The best framework for a fast and environmentally sustainable website is…

Blog

Author
By Josh Stopper
Published
November 29, 2023

Despite claims by the teams behind frameworks, or an agency trying to sell you a product, no framework in isolation can guarantee you a faster or environmentally sustainable website. A fast and environmentally sustainable website is the result of a concerted effort with an unwavering focus.

In the world of web development, the browser wars have largely been replaced by the framework wars. All too often, like the browser wars, the discussions on frameworks quickly fall prey to not just one logical fallacy, but several.

“Logical fallacies are deceptive or false arguments that may seem stronger than they actually are due to psychological persuasion, but are proven wrong with reasoning and further examination.

These mistakes in reasoning typically consist of an argument and a premise that does not support the conclusion.”

https://blog.hubspot.com/marketing/common-logical-fallacies

Whilst everyone is entitled to an opinion, it simply isn’t ok to misrepresent an opinion as fact (a subject the world has become much more aware of in the last few years). Drawing conclusions about the best framework for a fast and environmentally sustainable website ignores the simple fact that even the best framework can be used to produce a terrible result from an inexperienced, under-resourced, or underfunded project team. You might know this as the Project management triangle.

Whilst you were drawn in with the temptation of a revelation, I will not be crowning “the one true framework to rule them all”. In fact, I will be doing quite the opposite.

The Logical Fallacies

An all too common scenario in today’s web framework landscape: A new framework is released with bold claims like “Did we mention it’s fast?”, “rarely requires manual optimisation”, “Built-in automatic image, font, and script optimisations for improved UX and Core Web Vitals”. More often than not, these statements have to be taken at face value as they fall prey to the hasty generalisation fallacy. 

“The hasty generalization fallacy is sometimes called the over-generalization fallacy. It is basically making a claim based on evidence that is just too small. Essentially, you can’t make a claim and say that something is true if you have only an example or two as evidence.”

https://owl.excelsior.edu/argument-and-critical-thinking/logical-fallacies/logical-fallacies-hasty-generalization/

In a recent social media post (details have been removed to protect participants), a discussion on environmentally sustainable websites and the frameworks used to produce them, demonstrated how easy it is to make a hasty generalisation, and how dangerous the fallacy can be:

The last key point is about using XXX for your site. I’ve checked second from the bottom and they also use XXX. You’ve done a great job optimizing it, but XXX naturally uses things like XXX and lots of XXX. These don’t really change your Lighthouse score, but they do add extra data transfer that isn’t needed.

I also checked all the other sites and noticed that the ones with the best scores are using XXX. This shows that XXX, and probably XXX (Which is a framework based on XXX), is perfect from a sustainability point of view

Whilst an excerpt is rarely the best format to judge a conversation, in two short paragraphs, there are various generalisations and opinions suggested as fact, that upon a deeper look the arguments can be seen for what they are:

Whilst the reach of the conversation is not likely to be broad enough to have a significant impact in the wider conversation of website performance and environmental sustainability, if statements like the above are taken as fact by those who are less familiar with the subject, the conversation can, and most likely results in a negative outcome for those involved, possibly leading to the bandwagon fallacy.

As a note, the author of the comment more than likely intended for this to be a positive message, and their work in the field has been nothing to the contrary.

I forget myself, where are my manners facts

The HTTP Archive collects data, and reports on millions of websites at consistent intervals. Beginning in 2012, testing a mere 100,000 (no mean feat by the way), the HTTP Archive, for the month of October 2023, ran over 3.7 million tests on desktop, and over 5.5 million tests on mobile.

This data collected from the tests is reported upon in the HTTP Archive’s Core Web Vitals report. Among other things, the report allows for analysing trends in website performance and environmental sustainability, and the insights below are from the report generated for the month of October 2023. The reports can be viewed at the following urls:

By no means is the list of frameworks and technologies selected for the report comprehensive, nor the metrics reported on a full breakdown of the performance and environmental sustainability of the dataset.

Not so different after all!

The analysis reveals that on desktop, with over 3.7 million tests, there is an average 8.4% standard deviation in the average Core Web Vitals scores. On mobile, with over 5.5 million tests, there is little more than an average 10.7% standard deviation in average Core Web Vitals scores.

What this means is that there is only a maximum 10.7% difference on average in performance despite the various technologies and frameworks used.

Core web vitals contribute to Google’s search algorithm, and when you consider that less than 45% of sites are passing on mobile and just over 50% of sites are passing on desktop, it’s evident that no technology is a shining beacon of hope for those looking for a quick way to achieve a passing score, let alone do well!

When it comes to environmental sustainability, the reports show an average difference in transfer size of 24.9% (584kb) on mobile, and 26.25% (714kb) on desktop. 700kb could be explained with one poorly optimised image or an eagerly loaded coded chat widget. 

The one true outlier (within the reported upon data) for sustainability are sites built with SvelteKit, as the median transfer size was well under the average median transfer size, coming in just under 50% smaller at 1260kb on mobile. The caveat to this conclusion (and if not acknowledged would be a hasty generalisation) is that of the ~3.7 million sites reported on for desktop, and ~5.5 million on mobile, SvelteKit only accounts for 2367 (0.06%) and 2462 (0.04%) of the tests respectively.

When measured against the recently implemented standardised rating system for website carbon emissions that we (beleaf) proposed to the sustainable web design committee, it is clear that there is no defined winner either. The rating system is based on transfer size percentiles from the entire HTTP Archive dataset for the month of June 2023. What this means is that at least 50% of tests are receiving an F rating. With the trend of transfer sizes increasing, the quantity of sites failing is set to increase as well.

To reiterate, no framework or technology is blazing a path to a fast or environmentally sustainability website.

So what framework should be used then?

I make no effort to conceal the fact that the majority of my work is built with WordPress. Nor do I delude myself with thoughts like “Everything should be built on WordPress”. My career to this point has resulted in the vast majority of my work needing a user friendly (opinion) content management system that solves the needs of the client, and a frontend that is not overly reactive to user input (no need for client side rendering like react/vue/angular/etc).

In conversations, I can come across as bullish on emerging and established frameworks that claim to improve the developer experience, or solve problems for developers better than another technology. For me, the question always comes back to “how does technology X solve the business problems of my client better than Y?”. For the most part, I believe that any technology can be used to achieve a positive outcome, and that the result is largely determined by the capabilities and experience of the team producing the work, and the resources made available to complete it.

No framework is going to be a quick solution for a fast and/or environmentally sustainable site. There are many contributing factors, some of them technical, some of them not. At the core of achieving a positive outcome for a project is the willingness to invest from the team involved in the production, and the client by making performance and environmental sustainability a core part of the deliverables.

When establishing the requirements or acceptance criteria for a new site, instead of starting the conversation with discussions of a certain technology or framework, or getting railroaded by “marketing speak”, consider defining quantifiable metrics that can (and should!) be validated, not just for the project you are embarking on, but against the past work by the partner you are considering working with.

Beleaf was founded on the principle of delivering best in class work across performance, sustainability, and accessibility. If you want to see how our sites do, have a look at our page in The Index. If you would like to work on a project with us, or if you would simply love to have a chat about performance or environmental sustainability, and how the work we do can and has an impact, reach out to Rob or myself.

Appendix

Content arguably plays the biggest part in emissions

Simply putting all the blame on the code of the website when it comes to emissions is incorrect. If a page with rich media items (images, videos, audio files) are loaded on the page, it is most likely going to have a larger transfer size than a page without.

Caveats in sustainability and performance reports

The sustainable web design methodology for calculating emissions uses hardcoded metrics for various parts of the formula based on information available in scientific literature. Critically for the point of this article, the energy required, and therefore the emissions produced to generate the HTML of the page requested, before transmitting it to the end user device, is considered a constant, and does not account for local energy intensities.

The simple fact here is that the energy required by a server to respond to a request is much lower if there is no server side processing required to generate the page. In simpler terms, a static HTML file compared to a server side rendered HTML document prepared with PHP, Javascript, Ruby, or otherwise.

In a 2017 study titled the Energy Efficiency across Programming Languages, the conclusion of which language to use changed depending on different sets of objectives. Whilst the tests completed to generate the data exactly correlate with the average web developers day, the study highlights the challenges in choosing “the best” language when considering “time, memory, and energy”. This is all before the constraints and requirements of the business objectives are added to the mix. 

What the study alluded to is that it’s more important what you are doing than the technology you do it with. A page querying a database to present a table of data will be infinitely more efficient if there is a level of caching between the query and the database.

Important to note that even if a static HTML file is used, the TTFB is going to fluctuate due to other factors such as distance between the user and the data centre, as well as network congestion on the route the data takes between the user and the data centre (this is not an exhaustive list, merely two critical factors).