Systematic Literature Reviews (SLRs) are vital for advancing research, but they often feel like an uphill battle—demanding immense time and resources while delivering limited industry impact. What if we could make SLRs more efficient, impactful, and easier to sustain? Explore the transformative concept of sustainable SLRs and discover how a new approach can revolutionize research practices!

What Does Sustainability Mean for Systematic Literature Reviews?

Systematic Literature Reviews (SLRs) are a cornerstone of software engineering research. They help summarize the findings of numerous studies and identify gaps in knowledge. But as useful as they are, conducting an SLR can be overwhelming — they’re time-consuming, resource-intensive, and often fail to deliver practical insights that resonate with the industry.

So, is it time to rethink how we approach SLRs? Let’s dive into the concept of Sustainability of Systematic Literature Reviews — a new perspective that aims to make SLRs more efficient, impactful, and valuable.

Why Do SLRs Need Sustainability?

SLRs are not without their challenges. Some of the most common problems include:

1. High Resource Consumption: Planning, conducting, and updating SLRs demands a huge amount of time and effort.
2. Poor Documentation: Many SLRs lack proper documentation, making them difficult to replicate or update.
3. Low Industry Impact: Despite their rigorous methods, SLR results often fail to influence industry practices.
4. Lack of Quality: Problems with methodology, data synthesis, and reporting can undermine the credibility of SLRs.

These issues have led many researchers to question whether the effort invested in SLRs is worth it. This is where the idea of sustainability comes in.

What Is Sustainability in SLRs?

Sustainability in the context of SLRs is a holistic concept borrowed from environmental science and software engineering. In short, it means that SLR should be: (i) easier to sustain over time; (ii) consume fewer resources (time and effort); (iii) Deliver actionable insights for both academia and industry; (iv) Are easy to conduct, replicate, and update.

Dimensions of Sustainability in SLRs

Considering the multifaceted nature of SLR challenges, sustainability is broken into three dimensions:

1. Social Dimension

This addresses the human aspects associated with SLRs, such as communication, collaboration, and ensuring the effective participation of stakeholders. It emphasizes providing open access to SLR results, ensuring they are comprehensible and usable by both the academic community and industry practitioners.

2. Economic Dimension

This focuses on preserving resources, such as time and effort, during the planning, execution, and updating of SLRs. It emphasizes the importance of high-quality documentation, credible databases, efficient search engines, and adherence to rigorous guidelines to ensure reproducibility and minimize unnecessary efforts.

3. Technical Dimension

This dimension highlights the reliability and integration of tools used to conduct SLRs. By automating tasks and ensuring the technical feasibility of reconducting or updating SLRs, it aims to minimize manual labor and enhance process efficiency.

The intersection of these dimensions defines sustainability as: “The process and a set of actions to make it possible to preserve SLRs that endure over time (i.e., longevity) with the least possible resource consumption while delivering impactful results.”

Graphical Representation of Sustainability in SLRs

It is still a challenge to comprehend and communicate the interconnected nature of sustainability dimensions, researchers often use graphical models. Inspired by the Brundtland Report and other studies, the Venn diagram is a common representation.

Balanced View

Figure A represents a balanced view where all three dimensions intersect equally. While this approach ensures no single dimension dominates, its feasibility is often questioned due to the complexity of balancing diverse factors.

Unbalanced View

Figure C proposes an unbalanced view whose technical dimension is a common concern interwoven with social and economic aspects. This suggests that we can not dissociate technical concerns from social and economic aspects. Nevertheless, it is still possible to have problems that specifically attend to social or economic dimensions.

Nested Models

Figures (B) and (D) arrange the dimensions hierarchically suggesting that researchers should consider different weightings and priorities. For instance, technical concerns should only be treated whenever social and economic perspectives are already considered.

Despite their differences, these models are the first glimpse of sustainability of SLR. They highlight the need for integrated thinking and collaboration to achieve truly sustainable SLRs.

A New Mindset for SLRs

The idea of sustainability in SLRs isn’t about solving every single problem. It’s about encouraging researchers to adopt a more holistic view. Instead of tackling issues in isolation, we need to think about how problems, their causes, and their solutions are interconnected.

For instance, poor documentation doesn’t just affect reproducibility. It also impacts trust in the results, which can reduce their adoption by the industry. Similarly, using better tools can reduce time and effort but also improve the quality and reliability of findings.

By addressing these challenges from multiple angles — social, economic, and technical — we can create a more sustainable approach to SLRs.

Why Should You Care About Sustainable SLRs?

If you’re a researcher, sustainable SLRs can save you a ton of time and frustration while producing results that are more impactful and easier to share. If you’re an industry practitioner, sustainable SLRs mean you’ll finally get insights that are practical and directly applicable to your work.

In short, sustainability makes SLRs better for everyone.

Final Thoughts

SLRs are invaluable tools for advancing knowledge in software engineering, but they’re long overdue for an upgrade. By embracing the principles of sustainability, we can transform SLRs from a resource-heavy, isolated activity into a streamlined, collaborative process that delivers real value.

We believe this paper provides a good starting point to raise awareness about the critical need for sustainable practices in SLRs. Addressing social, economic, and technical dimensions in an integrated way is essential for producing SLRs that are efficient, impactful, and enduring.

It’s time to shift our mindset and make SLRs a sustainable endeavor. The future of research depends on it.

What are your thoughts on this? Have you faced any challenges with conducting or using SLRs?

Read the full paper ‘Towards Sustainability of Systematic Literature Reviews’.