Survey data appendix.
Interactive online appendix for “Tool Adoption in Systematic Literature Reviews”. It reports survey responses across 8 SLR phases, covering workflow fragmentation, AI adoption, and integration experience (N = 178, December 2025 to February 2026).
Methodological guidelines for systematic literature reviews (SLRs) specify how to search, screen, and synthesize evidence, but they rarely address the digital tools that carry the work. The range of available tools has grown, from reference managers and dedicated screening platforms to AI-assisted applications. Yet how researchers select, combine, and coordinate them across a review remains empirically underexplored.
The study surveys N = 178 researchers with recent SLR experience, recruited from the information systems conference community (ICIS, ECIS, AMCIS, PACIS, HICSS, WI), on their tool use across eight workflow activities. Two questions guide it. RQ1 asks what digital tools researchers employ across SLR activities. RQ2 asks what factors shape tool choice and how researchers experience the coordination of multi-tool workflows. Rather than comparing tool features, the study treats reviews as multi-activity digital workflows and shifts the unit of analysis from individual tools to respondent-specific configurations.
Key findings
What the survey shows, at a glance
Spreadsheets persist.
Excel / Google Sheets is the most-used tool in five of six applicable activities, peaking at 83.71% in data extraction. It is used by 92.13% of respondents in at least one activity, even where dedicated alternatives exist.
Workflows are fragmented.
The 178 respondents form 122 distinct three-activity tool stacks. Of these, 80% are reported by a single respondent, and the most common configuration covers under 4%. There is no standard setup.
Coordination is the bottleneck.
Over three quarters (77.53%) report integration difficulty. Lack of integration is the most-cited challenge (50.56%), and cross-tool coordination is the lowest-rated performance dimension (47.19%).
AI is concentrated in writing.
A majority (55.62%) use at least one AI tool, but mostly for writing (44.38% general LLMs). Screening, extraction, and analysis remain dominated by non-use. Users value efficiency, yet only 14.43% trust outputs, and 92.55% verify them.
Methodology
Study design, sampling, and survey instrument
We conducted an exploratory survey to map current digital tool practices among researchers who have conducted systematic literature reviews (SLRs). Respondents were asked to report on the most recent SLR they had conducted. The survey combined closed-ended questions on tool use, selection criteria, perceived performance, and institutional context with open-text questions on challenges, surprises, desired features, and disclosure practices.
We developed the survey instrument based on prior work on SLR tools, research workflows, and AI-assisted review practices. It was piloted with five researchers and revised accordingly. The final version comprises 38 items in eight sections: (1) review context, (2) tool use across eight SLR activities, (3) AI-assisted tool use and perceived impact, (4) tool selection factors, (5) tool performance perceptions, (6) institutional context and integration experience, (7) challenges, documentation practices, and open-text reflections, and (8) respondent demographics.
We used convenience sampling with multiple recruitment channels. The primary channel consisted of personalized email invitations to authors of SLR-related papers published at major IS conferences (ICIS, ECIS, AMCIS, PACIS, HICSS, and WI) between 2022 and 2026. Candidates were identified through the AIS eLibrary using the search term “systematic literature review” in titles, abstracts, or subject fields. Recruitment was complemented through departmental networks at the authors' institutions.
The survey was open for ten weeks (December 2025 to February 2026).
| Stage | N | % of previous |
|---|---|---|
| Email invitations sent | 1,352 | — |
| Undeliverable / errors | 112 | 8.28% |
| Effective invitations | 1,240 | — |
| Responses received | 268 | 21.61% |
| Complete responses | 181 | 67.54% |
| Confirmed SLR experience (analytical sample) | 178 | 98.34% |
Closed-ended items were analyzed using descriptive statistics (frequencies, percentages, means, medians). Open-text responses from six fields (total coded responses n = 434) were analyzed using inductive thematic coding (Braun & Clarke, 2006). To assess workflow fragmentation, we analyzed respondent-specific tool stacks across reference management, screening, and data extraction. These three activities were selected because they form a sequence of closely connected transitions in which records are organized, filtered, and transformed into a structured evidence base.
38 items across 8 sections. Items include multi-select questions, Likert-type scales, and open-text fields.
View full survey instrument
Section 1: Consent & Eligibility
- Have you conducted a systematic literature review in the last 5 years? (Yes/No, filter)
Section 2: Most Recent Review
- Number of databases/sources systematically searched (1 / 2–3 / 4–5 / 6+)
- Number of researchers involved (numeric)
- Screening approach (single reviewer / spot-checks / divided / dual independent / dual with calibration / AI-assisted with verification)
- Initial search results: approximate total records (numeric)
- Final included studies (numeric)
- Manuscript status (published / under review / completed not submitted / in progress / not intended)
- Duration (<3 months / 3–6 / 6–12 / >12 / ongoing)
Section 3: Digital Tools Used (multi-select per phase)
- Literature Search: Database websites, Google Scholar, Specialized tools, API/Programming, AI-supported tools, Other
- Reference Management: Zotero, Mendeley, EndNote, Citavi, JabRef, Excel/Sheets, BibTeX/LaTeX, Notion/Airtable, No dedicated tool, Other
- Screening: Covidence, Rayyan, EPPI-Reviewer, Abstrackr, ASReview, Excel/Sheets, Ref. mgmt. software, Notion/Airtable, No structured tool, Other
- Data Extraction: Excel/Sheets, Screening tool, Dedicated database, Qualitative software, Word/Docs, Other
- Qualitative Analysis: NVivo, MAXQDA, Atlas.ti, Excel/Sheets, Word markings, No dedicated software, Other
- Quantitative Analysis: Not applicable, R, SPSS, Stata, RevMan, CMA, Excel, Python, Other
- Visualization: Excel/Sheets, R (ggplot2), Python (matplotlib), Tableau/Power BI, Diagram tools, PowerPoint/Keynote, VOSviewer, Other
- Writing & Collaboration: Microsoft Word, Google Docs, LaTeX/Overleaf, Notion/Confluence, No collaboration tools, Other
Section 4: AI-Assisted Tools
- AI tool usage matrix: phases (search, screening, extraction, analysis, writing) × tool types (General LLMs, Research AI, AI Screening, Other AI, No usage)
- How AI tools were used (free text, conditional)
- AI impact: 8 Likert items (efficiency, quality, complemented tools, caused problems, benefits unclear, trust, verification, recommendation)
- Overall AI usage statement (replaced / complemented / exploratory / stopped / other)
Section 5: Tool Selection & Fit
- Selection factors: 10 items, importance scale 1 to 5 (free/open-source, institutional license, familiarity, colleague rec., supervisor rec., methodological literature, functionality, ease of use, collaboration features, speed/efficiency)
- Performance assessment: 8 items, agreement scale 1 to 5 (suited to task, efficiency, functional coverage, ease of use, interoperability, reliability, would reuse, overall satisfaction)
Section 6: Institutional Context
- Institutional influences (multi-select: licenses, IT support, training, standard in group, budget, policy, no influence)
- Tool integration experience (seamless / manual but feasible / isolated high effort / major problems / N/A)
Section 7: Challenges & Documentation
- Tool-related challenges (multi-select: costs, learning curve, integration, collaboration, data volume, unclear standards, time, missing features, none)
- Most significant struggle or frustration (free text)
- Documentation of tools in manuscript (yes fully / partially / upon request / no / not finished)
- Documentation of AI tool usage (yes fully / partially / deliberately not / didn't think of it / not decided)
- Reasoning for AI documentation decision (free text)
- Surprises: tools/workflows that worked well or poorly (free text)
Section 8: Background
- Career stage (student / PhD / postdoc / asst. prof / assoc. prof / full prof / other)
- Total review experience count (numeric)
- Desired tools/features wishlist (free text)
- Primary research field (IS, CS, Medicine, Engineering, Social Sciences, Psychology, Economics, etc.)
- Additional comments (free text)