Validity Issues in the Use of Social Network Analysis with Digital Trace Data
Journal of the Association for Information SystemsPublished 1 December 2011
James Howison, Andrea Wiggins, Kevin Crowston
Citations352
SJR quartileQ1
SJR score3.12
SNIP2.36
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Ten validity issues associated with the combination of digital trace data and social network analysis methods are examined, with examples from the IS literature, to provide recommendations for improving the validity of future research.
Abstract
There is an exciting natural match between social network analysis methods and the growth of data sources produced by social interactions via information technologies, from online communities to corporate information systems. Information Systems researchers have not been slow to embrace this combination of method and data. Such systems increasingly provide “digital trace data” that provide new research opportunities. Yet digital trace data are substantively different from the survey and interview data for which network analysis measures and interpretations were originally developed. This paper examines 10 validity issues associated with the combination of digital trace data and social network analysis methods, with examples from the IS literature, to provide recommendations for improving the validity of future research.
Keywords
Social SciencesPhysics and Astronomy
The Constitution of Society. Outline of the Theory of Structuration
21,348 Citations1984Anthony Giddens
Experimental and quasi-experimental designs for generalized causal inference
13,404 Citations2010William R. Shadish, Thomas D. Cook +1 more
ScienceNetwork Analysis in the Social Sciences
4,229 Citations2009Stephen P. Borgatti, Ajay Mehra +2 more
The kinds of things that social scientists have tried to explain using social network analysis are reviewed and a nutshell description of the basic assumptions, goals, and explanatory mechanisms prevalent in the field is provided.
MIS QuarterlyWhy Should I Share? Examining Social Capital and Knowledge Contribution in Electronic Networks of Practice1
3,881 Citations2005Molly Wasko, Faraj
This study empirically test a model of knowledge contribution and finds that people contribute their knowledge when they perceive that it enhances their professional reputations, when they have the experience to share, and when they are structurally embedded in the network.
MIS QuarterlyWhy should i share? examining social capital and knowledge contribution in electronic networks of practice
3,560 Citations2005Molly Wasko, Samer Faraj
Social NetworksNode centrality in weighted networks: Generalizing degree and shortest paths
3,527 Citations2010Tore Opsahl, Filip Agneessens +1 more
This paper proposes generalizations that combine tie strength and node centrality, and illustrates the benefits of this approach by applying one of them to Freeman’s EIES dataset.
Social NetworksCentrality and network flow
3,061 Citations2005Stephen P. Borgatti
A key claim made in this paper is that centrality measures can be regarded as generating expected values for certain kinds of node outcomes given implicit models of how traffic flows, and that this provides a new and useful way of thinking about centrality.
SSRN Electronic JournalStructural Holes: The Social Structure of Competition
3,037 Citations1992Ronald S. Burt
Journal of ManagementThe Network Paradigm in Organizational Research: A Review and Typology
2,948 Citations2003Stephen P. Borgatti
Information Systems ResearchResearch Commentary: Desperately Seeking the “IT” in IT Research—A Call to Theorizing the IT Artifact
2,722 Citations2001Wanda J. Orlikowski, Calogero Iacono
It is proposed that IS researchers begin to theorize specifically about IT artifacts, and then incorporate these theories explicitly into their studies, and believe that such a research direction is critical if IS research is to make a significant contribution to the understanding of a world increasingly suffused with ubiquitous, interdependent, and emergent information technologies.
Graphs over time
2,264 Citations2005Jure Leskovec, Jon Kleinberg +1 more
A new graph generator is provided, based on a "forest fire" spreading process, that has a simple, intuitive justification, requires very few parameters (like the "flammability" of nodes), and produces graphs exhibiting the full range of properties observed both in prior work and in the present study.
Oxford University Press eBooksTheories of Communication Networks
2,153 Citations2003Peter R. Monge, Noshir Contractor
MultiTheoretical, Multilevel Models of Communication and Other Organizational Networks Appendix: Data Sets Used in Chapter 2 References Author Index Subject Index
Cambridge University Press eBooksExploratory Social Network Analysis with Pajek
2,083 Citations2018Wouter de Nooy, Andrej Mrvar +1 more
Annual Review of SociologyNetwork Data and Measurement
2,035 Citations1990Peter V. Marsden
Continued research on data quality is needed; beyond improved samples and further investigation of the informant accuracy/reliability issue, this should cover common indices of network structure, address the consequences of sampling portions of a network, and examine the robustness of indicators ofnetwork structure and position to both random and nonrandom errors of measurement.
Information Systems ResearchImprovising Organizational Transformation Over Time: A Situated Change Perspective
1,936 Citations1996Wanda J. Orlikowski
A perspective on organizational transformation is outlined which proposes change as endemic to the practice of organizing and hence as enacted through the situated practices of organizational actors as they improvise, innovate, and adjust their work routines over time.
ScienceEmpirical Analysis of an Evolving Social Network
1,743 Citations2006Gueorgi Kossinets, Duncan J. Watts
This work analyzed a dynamic social network comprising 43,553 students, faculty, and staff at a large university, in which interactions between individuals are inferred from time-stamped e-mail headers recorded over one academic year and are matched with affiliations and attributes.
Centrality and network flow
1,739 Citations2005Stephen P. Borgatti
Research PolicyHow open source software works: “free” user-to-user assistance
1,597 Citations2003Karim R. Lakhani, Eric von Hippel
Social Networks and Organizations
1,553 Citations2003Martín Kilduff, Wenpin Tsai
The Journal of Strategic Information SystemsSome antecedents and effects of trust in virtual communities
1,404 Citations2002Catherine M. Ridings, David Gefen +1 more
Trust was increased through perceived responsive relationships in the virtual community, by a general disposition to trust, and by the belief that others confide personal information.
Journal of Computer-Mediated CommunicationStudying Online Social Networks
1,105 Citations2006Laura Garton, Caroline Haythornthwaite +1 more
The utility of the social network approach for studying computer-mediated communication, be it in computer-supported cooperative work, in virtual community, or in more diffuse interactions over less bounded systems such as the Internet is shown.
Social NetworksClustering in weighted networks
1,026 Citations2009Tore Opsahl, Pietro Panzarasa
This paper focuses on a measure originally defined for unweighted networks: the global clustering coefficient, and proposes a generalization of this coefficient that retains the information encoded in the weights of ties.
Analyzing Social Media Networks with NodeXL: Insights from a Connected World
1,001 Citations2010Derek L. Hansen, Ben Shneiderman +1 more
Analyzing Social Media Networks with NodeXL walks readers through using NodeXL while explaining the theory and development behind each step, providing takeaways that can apply to any SNA and demonstrates how visual analytics research can be applied to SNA tools for the mass market.
Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topicsScale-free topology of e-mail networks
952 Citations2002Holger Ebel, Lutz-Ingo Mielsch +1 more
The resulting network exhibits a scale-free link distribution and pronounced small-world behavior, as observed in other social networks, implying that the spreading of e-mail viruses is greatly facilitated in real e- mail networks compared to random architectures.
Centrality in Social Networks
761 Citations1993Vladimir Batagelj
In the paper an introduction to main social networks centrality measures is given, a new view on these measures is proposed, based on relational algebra, which is implemented in computer programs CENTRAL and FLOWIND.
Organizations and Communication Technology
727 Citations1990Janet Fulk, Charles Steinfield
Social NetworksOn the robustness of centrality measures under conditions of imperfect data
722 Citations2005Stephen P. Borgatti, Kathleen M. Carley +1 more
Dense networks were the most robust in the face of all kinds of error except edge deletion, which suggests that, for random networks and random error, the authors shall be able to construct confidence intervals around centrality scores.
Lecture notes in computer sciencePajek— Analysis and Visualization of Large Networks
717 Citations2002Vladimir Batagelj, Andrej Mrvar
Social NetworksCentrality in social networks: ii. experimental results
712 Citations1979Linton C. Freeman, Douglas Roeder +1 more
Management ScienceDoes Information Technology Lead to Smaller Firms?
694 Citations1994Erik Brynjolfsson, Thomas W. Malone +2 more
Broad evidence is found that investment in IT is significantly associated with subsequent decreases in the average size of firms, and that these decreases in firm size are most pronounced two to three years after the IT investment is made.
Social NetworksHow to search a social network
612 Citations2005Lada A. Adamic, Eytan Adar
It is found that small world search strategies using a contact’s position in physical space or in an organizational hierarchy relative to the target can effectively be used to locate most individuals, but in the online student network, where the data is incomplete and hierarchical structures are not well defined, local search strategies are less effective.
Mining email social networks
558 Citations2006Christian Bird, Alex Gourley +3 more
This paper begins with a discussion of the infrastructure (including a novel use of Scientific Workflow software) and then discusses the approach to mining the email archives, and presents some preliminary results from the data analysis.
Organization ScienceNetwork Structure in Virtual Organizations
507 Citations1999Manju Ahuja, Kathleen M. Carley
Management ScienceLocation, Location, Location: How Network Embeddedness Affects Project Success in Open Source Systems
504 Citations2006Rajdeep Grewal, Gary L. Lilien +1 more
This work finds that considerable heterogeneity exists in the network embeddedness of open source projects and project managers, and uses latent class regression analysis to show that multiple regimes exist and that some of the effects ofnetwork embeddedness are positive under some regimes and negative under others.
Analyzing (social media) networks with NodeXL
500 Citations2009Marc A. Smith, Ben Shneiderman +7 more
A sequence of NodeXL operations from data import to computation of network statistics and refinement of network visualization through sorting, filtering, and clustering functions is described, revealing sociologically relevant differences in the patterns of interconnection among employee participants in the social media space.
American Journal of SociologyDynamic Network Visualization
493 Citations2005James Moody, Daniel A. McFarland +1 more
The authors divide network movies into static flip books, where node position remains constant but edges cumulate over time, and dynamic movies, where nodes move as a function of changes in relations.
Journal of Statistical SoftwareSocial Network Analysis with<b>sna</b>
459 Citations2008Carter T. Butts
An overview of a software package which provides support for a range of network analytic functionality within the R statistical computing environment is provided.
Communications of the ACMThe convergence of social and technological networks
347 Citations2008Jon Kleinberg
Internet-based data on human interaction connects scientific inquiry like never before and helps scientists understand the world around us more fully.
NatureA twenty-first century science
326 Citations2007Duncan J. Watts
If handled appropriately, data about Internet-based communication and interactivity could revolutionize the understanding of collective human behaviour.
Information Systems ResearchThe Effects of the Social Structure of Digital Networks on Viral Marketing Performance
311 Citations2008Mauro Bampo, Michael T. Ewing +3 more
This paper deconstructs the process and investigates the formation of the activated digital network as distinct from the underlying social network, confirming that the social structure of digital networks play a critical role in the spread of a viral message.
Socio-technical congruence
302 Citations2008Marcelo Cataldo, James D. Herbsleb +1 more
This paper argues that modularization, the traditional technique intended to reduce interdependencies among components of a system, has serious limitations in the context of software development and builds on the idea of congruence, proposed in prior work, to examine the relationship between the structure of technical and work dependencies.
Information Systems ResearchEmergence of New Project Teams from Open Source Software Developer Networks: Impact of Prior Collaboration Ties
292 Citations2008Jungpil Hahn, Jae Yun Moon +1 more
Overall, it is found that a developer is more likely to join a project when he has strong collaborative ties with its initiator, and perceived status of the noninitiator members of a project influences its probability of attracting developers.
Recent Patents on EngineeringRecent Advances of Finite Elements for Laminated Composite Plates
232 Citations2008Yixia Zhang, Chunhui Yang +2 more
Trace Ethnography: Following Coordination through Documentary Practices
222 Citations2011R. Stuart Geiger, David Ribes
The methodology of 'trace ethnography', which combines the richness of participant- observation with the wealth of data in logs so as to reconstruct patterns and practices of users in distributed sociotechnical systems, is detailed.
Sociological Methods & ResearchStatistical Analysis of Longitudinal Network Data With Changing Composition
180 Citations2003Mark Huisman, Tom A. B. Snijders
The authors propose an extension of the simulation algorithm of stochastic actor-oriented models for network change to include networks of changing composition, which is based on the simulation of Markov chains.
Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)PCANS Model of Structure in Organizations
174 Citations2018David Krackhardt, Kathleen M. Carley
The utility of the network based approach to characterize organizational architectures in terms of individuals, tasks, and resources is demonstrated for re-characterizing classical organizational theory and for generating a series of testable hypotheses about organizational performance.
Knowledge, technology & policy/Knowledge in society/Knowledge, technology, & policyHierarchy and centralization in free and open source software team communications
171 Citations2006Kevin Crowston, James Howison
It is found that some OSS teams are highly centralized, but contrary to expectation, others are not; and the level of centralization is negatively correlated with project size, suggesting that larger projects become more modular, or possibly that becoming more modular is a key to growth.
Information Systems Research<b>Editorial Overview</b>—The Interplay Between Digital and Social Networks
165 Citations2008Ritu Agarwal, Anil K. Gupta +1 more
10 papers in this special issue of Information Systems Research shed light on the technical, behavioral, and economic challenges and implications of social networks and contribute to the understanding of how the power of such networks can be harnessed.
Gabler Verlag eBooksHow Open Source Software Works: “Free” User-to-User Assistance
155 Citations2004Karim R. Lakhani, Eric von Hippel
Organizational Behavior and Human Decision ProcessesOrganizational network perceptions versus reality: A small world after all?
150 Citations2008Martín Kilduff, Craig Crossland +2 more
Information Systems ResearchCasting the Net: A Multimodal Network Perspective on User-System Interactions
135 Citations2008Gerald C. Kane, Maryam Alavi
Relying heavily on the theory and methods behind social network analysis, the concept of multimodal networks is introduced to assess both users and information systems as equivalent nodes in a single social network.
Information Systems ResearchThe Impact of Information Diffusion on Bidding Behavior in Secret Reserve Price Auctions
134 Citations2008Oliver Hinz, Martin Spann
An analytical model for the effect of shared information on individual bidding behavior in a secret reserve price auction with a single buyer facing a single seller similar to eBay's Best Offer and some variants of NYOP is developed.
International Journal of Man-Machine StudiesComputer-mediated communication system network data: theoretical concerns and empirical examples
115 Citations1990Ronald E. Rice
The review combines two separate foci in recent research: the diffusion and use of computer-mediated communication systems in organizations, and the conceptualization of communication as a process of interaction and convergence, as represented by the network paradigm.
Health Services ResearchValidity of Measures Is No Simple Matter
99 Citations2005Lee Sechrest
This article aims to promote a better understanding of the nature of measurement, the special problems posed by measurement in the social sciences, and the inevitable limitations on inferences in science by using the measurement of blood pressure as an example.
IFIP International Federation for Information Processing/IFIPSocial dynamics of free and open source team communications
94 Citations2006James Howison, Keisuke Inoue +1 more
It is provided empirical evidence that while change at the center of FLOSS projects is relatively uncommon, participation across the project communities is highly skewed, with many participants appearing for only one period.
Journal of Database ManagementSocial Network Structures in Open Source Software Development Teams
92 Citations2007Yuan Long, Keng Siau
The finding suggests that the interaction pattern of OSS projects evolves from a single hub at the beginning to a core/periphery model as the projects move forward.
Information Systems Research<b>Research Note</b>—Towards Dynamic Visualization for Understanding Evolution of Digital Communication Networks
91 Citations2008Matthias Trier
This paper presents a method for event-based dynamic network visualization and analysis together with its exploratory social network intelligence software Commetrix and demonstrates how exploration of animated graphs combined with measuring temporal network changes identifies measurement artifacts of static network analysis.
Computational and Mathematical Organization TheoryRobustness of centrality measures under uncertainty: Examining the role of network topology
89 Citations2009Terrill L. Frantz, Marcelo Cataldo +1 more
The experiments show that the accuracy of identifying the prestigious, or key, actors in a network—according observed data—is considerably predisposed by the topology of the ground-truth network.
Syracuse University Libraries (Syracuse University)The S ocial Structure of Open Source Soft ware Development Teams
74 Citations2003Kevin Crowston, James Howison
It is found that some OSS teams are highly centralized, but contrary to expectation, others are not, and it is found that the level of centralization is negatively correlated with project size, suggesting that larger projects become more modular.
Organizational Research MethodsAuditing Information Structures in Organizations: A Review of Data Collection Techniques for Network Analysis
63 Citations2005Karen H. Zwijze‐Koning, Menno D.T. de Jong
This article reviews the methodological literature on methods that may be used for network analysis and discusses reliability and validity issues, as well as the applicability for organizational research questions.
Complex Network Measurements: Estimating the Relevance of Observed Properties
59 Citations2008Matthieu Latapy, Clémence Magnien
This work provides the first practical methodology to distinguish between cases where the assumption that samples large enough to be representative of the whole are misleading, and casesWhere the observed properties may be trusted.
SSRN Electronic JournalSocio-Technical Congruence: A Framework for Assessing the Impact of Technical and Work Dependencies on Software Development
58 Citations2008Marcelo Cataldo, James D. Herbsleb +1 more
IEEE Intelligent SystemsSelf-Organization Patterns in Wasp and Open Source Communities
54 Citations2006Sergi Valverde, Guy Théraulaz +3 more
A comparative study of how social organization takes place in a wasp colony and OSS developer communities is conducted, analyzing the patterns and functional significance of these systems' weighted-interaction networks.
AIS Electronic Library (AISeL) (Association for Information Systems)Studying Community Dynamics with an Incremental Graph Mining Algorithm
48 Citations2008Tanja Falkowski, Anja Barth +1 more
TheDENGRAPH is an incremental graph mining algorithm, which detects and adapts communities over time, and reports on the first results in applying DENGRAPH on the social network of mail interactions of ENRON.
Analysis of Activity in the Open Source Software Development Community
44 Citations2007Scott Christley, Greg Madey
This paper describes a data set of SourceForge.net, the world's largest open source software development site, which is available for research purposes, and discusses various data mining techniques that can be applied to the data and the type of research questions that could be answered.
AIS Electronic Library (AISeL) (Association for Information Systems)Lurking: Legitimate or Illegitimate Peripheral Participation?
42 Citations2006Adrian Yeow, Steven L. Johnson +1 more
It is concluded that lurking as legitimate or illegitimate peripheral participation is context-dependent and a more complex, nuanced activity than previously theorized and measured.
Academy of Management ProceedingsIt's a Network, Not an Encyclopedia: A Social Network Perspective on Wikipedia Collaboration.
39 Citations2009Gerald C. Kane
A Research Collaboratory for Open Source Software Research
38 Citations2007Yongqin Gao, Matthew Van Antwerp +2 more
The design and implementation of the research collaboratory and usage statistics for the year 2006 are described, including over two years of monthly database dumps from SourceForge.net.
Productivity Effects of Information Diffusion in Networks
35 Citations2007Sinan Aral, Erik Brynjolfsson +1 more
Network analysis of trace data for the support of group work
31 Citations2010Sean Goggins, Krista Galyen +1 more
A 16-student, completely online software design course was studied using social network analysis and grounded theory techniques, and it was indicated that bi-directional log data revealed a more accurate and complete description of small group activity than ordinary, uni-directionals log data would have.
A Social network approach to free/open source software simulation
31 Citations2005Patrick Wagstrom, Jim Herbsleb +1 more
This work has analyzed several methods of communication; a social networking site, project mailing lists, and developer weblogs; to gain an understanding of the social network structure behind F/OSS projects and create a model of F/OS development that allows for multiple projects, users, and developers with varying goals and socialization methods.
IFIP International Federation for Information Processing/IFIPSocial Dynamics of FLOSS Team Communication Across Channels
31 Citations2008Andrea Wiggins, James Howison +1 more
The validity of data sampling is evaluated by examining dynamics of communication centralization, which vary across multiple communication channels, and a method for intensity-based smoothing in dynamic social network analysis is introduced.
Validity of network analyses in Open Source Projects
25 Citations2010Roozbeh Nia, Christian Bird +2 more
This paper examines the stability of network metrics in the presence of inadequate and missing data on three different OSS projects and demonstrates that while these issues do change network topology, the metrics used in the analysis are stable with respect to such changes.
AIS Electronic Library (AISeL) (Association for Information Systems)Investigating Success of Open Source Software Projects: A Social Network Perspective
24 Citations2007Jing Wu, Khim Yong Goh +1 more
This research aims to analyze the survival of open source projects by using the theoretical lens of social network analysis, and finds significant impacts of communication patterns on project outcomes over the long term.
ProQuest LLC eBooksAlone together: a socio-technical theory of motivation, coordination and collaboration technologies in organizing for free and open source software development
16 Citations2009Kevin Crowston, James Howison
The dissertation concludes by enumerating the conditions under which this theory of organizing for self-organizing, virtual teams in other domains of work is likely to be successful, such as non-revokable and rewindable work with incremental incentives.
AIS Electronic Library (AISeL) (Association for Information Systems)Profile of the Social Network in Photo Sharing Systems
12 Citations2008Przemysław Kazienko, Katarzyna Musiał +1 more
The experiments on the Flickr photo sharing system revealed that users are inspired by both the semantic relationships between objects they operate on and social links they have to other users, suggesting the density and affluence of the social network grows over course of time.
2008 Proceedings IEEE INFOCOM - The 27th Conference on Computer CommunicationsComplex Network Measurements: Estimating the Relevance of Observed Properties
12 Citations2008M. Latapy, C. Magnien
Open Source Communities as Social Networks: An Analysis of Some Peculiar Characteristics
11 Citations2008Giulio Concas, Manuela Lisci +3 more
By examining developers mailing lists (MLs), some social network analysis (SNA) measures are adopted to evaluate interactions among community members to better understand communication flows among team members and whether there is someone coordinating, controlling and facilitating the information process.
AIS Electronic Library (AISeL) (Association for Information Systems)NETWORK EFFECTS IN OSS DEVELOPMENT : THE IMPACT OF USERS AND DEVELOPERS ON PROJECT PERFORMANCE
8 Citations2008Sherae L. Daniel, E. Ilana Diamant
Journal of the Association for Information SystemsAnalysis of Survival of Open Source Projects: a Social Network Perspective
7 Citations2007Jing Wu, Qian Tang
This research has studied the impact of the communication pattern of open source projects on the project success, defined in view of both the supply side and the demand side (end users), while incorporating control factors such as project-specific characteristics.
Academy of Management ProceedingsTHE CO-EVOLUTION OF SOCIAL NETWORKS AND SOFTWARE STRUCTURES: A STUDY OF OPEN SOURCE AND CLOSED SOURCE PROJECTS.
1 Citations2009Francesco Merlo, Sandra A. Slaughter +1 more
This study draws from the product development literature to hypothesize the symmetry between social networks and design structures in software projects and finds findings support this hypothesis but also refute this hypothesis.
…