A verdict about whether or not Chromium is more secure than Firefox
Two weeks ago, I made this post. The goal was simple: I wanted to dig into the details of Chromium and Firefox to see if the claims that Chromium is more secure than Firefox are true or not. You'll notice I also started turning that post into an update log, but only one update got released. There is a reason for that. Life suddenly got extremely busy for me, I could barely make time to continue researching. However, during that time, I spent a lot of time thinking about the issue. I tried breaking down the problem in a million different ways to find a way to simplify it and start from the ground up.
I came to a conclusion today, a realization. I have no way to put this gently: I cannot conclusively determine which one is more secure. This will upset many of you, and it upsets me too considering I maintain my own list of software that relies on only providing the most secure and private versions of some software. I need to explain why there cannot be a solid conclusion.
I managed to collect many sources to be used for the research. A lot of the information is parroting this article which, despite having many sources, fails to provide sources for some of the most crucial claims made there ("Fission in its current state is not as mature as Chromium's site isolation" has no source, for example). My favorite source is this Stanford paper which I think does a great job at tackling the problem. The problem I noticed is that a lot of privacy advice is given from an echo chamber.
Think about what privacy advice you like to give, and think about where you heard that. A YouTube video? Reddit? Lemmy? Naomi Brockwell gives a lot of advice that stems directly from Michael Bazzell's Extreme Privacy book, as I found out after reading it. Her videos about convincing people to use Signal are paraphrased passages from the book itself, which has a whole section about it. People touting Chromium as more secure than Firefox, or that the Play Store is a more secure option than F-Droid or Aurora Store, often get their information from GrapheneOS. I've never seen anyone research those in depth.
The point I'm trying to make is that a lot of privacy advice is circular reporting. I'm certain that if Michael Bazzell and GrapheneOS were to provide sources as to where they got their information (they rarely do, I checked) it would come to light that it boils down to a few real sources. GrapheneOS, no doubt, likely has inspected at least some part of the Firefox codebase, but Firefox is rapidly changing, so any sources that used to be true may not be true today.
FUTO Keyboard and GrayJay get recommended often because of Louis Rossmann, but HeliBoard and FreeTube (or NewPipe) were options long before those pieces of software. The reason the former became so recommended over the latter is simply because people used a popular figure, Louis Rossmann, as a primary source. It then became an echo chamber of recommendations and best practices.
That doesn't mean the claims of Chromium being more secure are false, but as a researcher it is very hard to credit something that doesn't provide any primary sources. In the eyes of a researcher, GrapheneOS's word holds just as much weight as a random internet user, without any proof. I see it play out like this: A source like GrapheneOS or Extreme Privacy makes a claim, secondary sources such as GrapheneOS users or Naomi Brockwell present this information without providing the sources, the general privacy community sees both, and begin giving the same recommendations on Reddit or Lemmy (sometimes with sources), and eventually the privacy community as a whole starts presenting that information, without any primary sources. Even if GrapheneOS, Extreme Privacy, or Louis Rossmann provided no research or direct comparisons, their word is taken without question and becomes the overarching recommendations in the privacy community. They each gained credibility in their own ways, but there should always be scrutiny when making a claim, no matter how credible.
The main reason why I cannot give a concrete conclusion is this: the focus on the article was to compare Chromium's Site Isolation to Firefox's implementation, however there are too many variables at play. Chromium may be more secure on one Linux distro than another. Debian is an example. Firefox supposedly has worse site isolation on Linux, but then how does Tails deal with that? It's based on Debian, so does that make it insecure for both browsers? Tor is based on Firefox ESR, which is an extended support release with less security, but Tor is also deemed a better option than Chromium browsers for anonymity. Isolating iframes doesn't really affect daily use, so is it really necessary to shame Firefox for that? Some variants of Firefox harden the browser for security, but some variants of Chromium (such as Brave Browser) try to enhance privacy. No matter what limits I set, how many operating systems or browser variants I set, there is no way to quantify which one is more secure.
"Is Chromium more secure? Yes, under XYZ conditions, with ABC variants, on IJK operating systems. Chromium variants XYZ are good for privacy, but ABC Firefox variants are better at privacy..." The article would be a mess. The idea for the article came because I was truly sick of the lack of true in-depth sources about the matter, and so I wanted to create that. I now realize it was a goal that is far too ambitious for me, or even a small group of people. Tor and Brave give different approaches to fingerprinting protection (blending in vs. randomizing), and there's no way to directly compare the two. The same goes for the security of each. There is no "Tails" for Chromium, but there is no "Vanadium" for Firefox. There's no one to one comparison for the code, because some of it is outside of the browser itself.
I regret making that initial post, because it set unrealistic expectations. It focused on a problem that can't tell the whole picture, and then promised to tell that whole picture. At a point, it comes down to threat model. Do you really need to squeeze out that extra privacy or security? Is someone going to go through that much effort? You know how to spot dark patterns, you know not to use privacy invasive platforms. Take a reality check. Both Chromium and Firefox are better than any proprietary alternatives, that's a fact. Don't bother trying to find the "perfect" Linux distro or browser for privacy and security, because you already don't use Windows. Privacy is a spectrum, and as long as you at least take some steps towards that, you've already done plenty.
Be careful next time you hear a software recommendation or a best practice. Be careful next time you recommend software or a best practice. Always think about where you heard that, and do your own research. There are some problems that are impossible or infeasible to solve, so just pick what you feel is best. I really am sorry that I wasn't able to provide what I promised, so instead I will leave a few of the sources I found helpful, just in case another ambitious person or group decides to research the matter. Not all of these sources are good, but it's a place to start:
GrapheneOS responded to my requests for a comment after this post was made, here: lemmy.ml/post/22142738
cvedetails.com/version-list/0/…
en.wikipedia.org/wiki/Site_iso…
madaidans-insecurities.github.…
news.ycombinator.com/item?id=3…
seclab.stanford.edu/websec/chr…
grapheneos.org/usage#web-brows…
reddit.com/r/browsers/comments…
wilderssecurity.com/threads/se…
forums.freebsd.org/threads/why…
GrapheneOS usage guide
Usage instructions for GrapheneOS, a security and privacy focused mobile OS with Android app compatibility.GrapheneOS
I am researching the claim that Chromium is more secure than Firefox
Edit: Here is the verdict: lemmy.ml/post/21887275I am currently doing a deep dive into whether or not Chromium is more secure than Firefox, and I will make a very long and comprehensive Lemmy post outlining my findings with specific sources. I expected this to take a few days, maybe a week, but after finding out many of the claims for both sides give no real sources, I expect this to take a month or longer. I will be reaching out to multiple first-party sources (Mozilla, GrapheneOS, etc.) to get their detailed statements on the matter. I want to provide something that actually covers the full picture of the issue with up to date sources, to hopefully put this to rest for anyone who doesn't want to do the research.
I'm making this post in case anyone wants to provide any extra resources they have about the issue. Do not fight about this issue in the comments, save that until after I am able to release my work. I'm tired of the constant back and forth about this with little to no direct sources. This means that my other project, Open Source Everything, will be put on pause. The FAQ section of that very project is what sparked this, because I realized the issue was far more complex than I outlined in there. (Don't trust the information in the FAQ just yet: it is still in the works.)
As always, don't just give blind support to this just because I am making promises, but if you feel your support is needed then by all means go for it.
If any of you want me to turn this post into an update log, let me know and I will.
DISCLAIMER: These update logs are NOT meant to be taken as a source. I am generalizing a lot of things here for simplicity and brevity, so do not try to pick it apart. Anything I say here is likely a summary of something that will be talked about in fine detail in the article, and so it may contain mistakes.
Update 1
I need to stop posting before bed, since I end up not being able to respond to drama quickly and it grows out of proportion. Anyways, I want to answer a few questions that keep popping up (maybe I'm obsessed with writing FAQs, I don't know) and then talk about my research process.Google Chrome is NOT the same as Chromium
This is something I already have a draft to write about in my article, because a lot of people mess up the distinction. Google Chrome is Google's proprietary "en-Googled" browser. That browser obviously has numerous privacy issues. What I am referring to in the article is what Google Chrome was built off of: Chromium. Chromium is open source (or source available, or something like that. Please stop trying to remind me of the difference, "open source" gets the point across). Many browsers such as Brave were built on top of Chromium. Many users in the privacy community use Chromium-based browsers. Chromium is mainly maintained by Google, but I will not be focusing on that since I am taking a look at the actual software and not any future problems that may arise.I'm summarizing things here, but I will go in depth in a section of my article about this, since a lot of people are still stuck on the mindset that Google is always evil. It is true that Google is bad with privacy, but they are good when it comes to security. They have to be, given that Chromium-based browsers and Android are the most used in their respective fields. Any privacy issues can be nullified with some projects like ungoogled-chromium or GrapheneOS which remove any privacy invasive Google components. Anything Google tries to sneak in doesn't get past those projects, like a safety net, because they take very close inspection of the code.
Security vs. Privacy
Security and privacy are two distinct topics with some overlap. As I mentioned above, any privacy issues can be dealt with by using some variants of the software. Because of this, my article will focus primarily on how secure these browsers are. I do understand that security and privacy can go hand in hand: Without security there is little privacy, and without privacy there is little security. However, that is all out of the scope of what I am researching here. The reason a lot of projects such as GrapheneOS recommend against Firefox browsers (especially on Android) is because they claim Firefox has weak site isolation. That is the main point of research for my article. If I can prove that those claims are true, I can demonstrate why it is such an issue. If I can prove that those claims are false, I can try to see if Firefox is more private than Chromium, and is therefor a better option. There will be other related ideas that will crop up that will be covered in the article, that I will research about. The broad hypothesis is "Chromium is more secure than Firefox" and it is my job to find out why people say that and investigate it.Also, many users talked about ad blocking and the recent removal of Manifest V2, which killed a lot of Chromium ad blockers. This is not the focus of the article, but let me remind you that using a browser such as Brave lets you block ads entirely. Brave is the only other browser recommended by the GrapheneOS project for its security, besides Vanadium. Yes, Brave has some bloat that can infringe on privacy, but those can be disabled. Don't forget that Brave is open source, so you are free to make a fork of it and remove whatever you'd like. The point is this: Both Chromium and Firefox both still have ad blocking, so this is a non-issue.
Who am I?
lemmy.ml/post/21367269/1428365…
first off, I have serious doubts that any one dude - or even a group of those for that matter - can ascertain the security of such a complex system; a browser is essentially an operating system, with all the layers and complexities that entails.even if you're somewhat successful in such an endeavor, I don't really care if it potentially is. chromium comes from those shitmakers and I'm not willingly using anything they had their nasty fingers in. they threw one shovel of shit too many on the heap and they are now forever on my ignore list. if that means that I don't get to access certain domains, sites, and/or apps - so be it, I'll make do without.
lemmy.ml/post/21367269/1428393…
Are you a single person or a group of people? Do you have any credentials that you'd like to share that might give some context to your research?Where is the quote in your bio from?
I could leave some cryptic retrospective answer here, and I would love to, but as fun as that would be it may cause more harm than good. I am an independent, singular person. If I were in your shoes, I too would doubt that any one person could research the intricacies of the matter. However, I don't need to look over every piece of code to make a conclusion. The main focus of the article, as I said, is site isolation. This is what most people reference when they talk about Chromium being "more secure" than Firefox. I already addressed the other argument about Chromium being "evil," as there are other projects that aim to remove some of the damage that has been done. Readers of my article will need to let down their precedent of Chromium being as bad as Google, and realize that Google is bad for privacy but good for security.If by "credentials" you mean actual identification, no. Even if I told you exactly who I was, you still would have no idea who I am. However, I can give you some of my background: I am advanced in the privacy field, proof of this can be seen with my other project. I used to work as a penetration tester for a low ranking government branch, focusing on network and website security. I am fluent in Python and C++, so I can understand a lot of the code that has been written. I hope that gives you context into who I am and what I do. I guess I could also mention I like to keep high standards, I'm a bit of a perfectionist. I want the article to be nothing short of extremely thorough and comprehensive.
The quote in my bio “Unjust laws only burden the just, as the lawless will not heed them.” is my own (hence why I put "- 8232" there). I have other quotes, but that one is my favorite.
How is the research going?
I didn't quite know where to start, but eventually I settled for this: I have three notes. One is for questions I have (e.g. "What is site isolation?") that I put answers under as I find them. This means I will never be trying to fill in the gaps without sources in the article. I'll have a well informed knowledge of everything. The next note is for all the sources about the issue, categorized into "Primary," "Secondary," and "Unverified" (when there is no source listed for the claim). The last notebook is people. This one contains people and groups who know about the issue that I may get statements or help from for the article. That is all I have right now, because I needed some sleep. I plan to add a "To-Do" note, some various drafts, and a list of documents about the issue. I'll keep this updated.An-anonymous-coder / Open-Source-Everything · GitLab
A curated list of the best open source software.GitLab
like this
Privacy reshared this.
kekmacska
in reply to The 8232 Project • • •BearOfaTime
Unknown parent • • •It also depends on your layering, or lack of. It's the complexity issue you ran into.
Great post by the way.
The 8232 Project
in reply to BearOfaTime • • •It leads me to, in the future, simply tell people to know how to stay safe no matter what they use. Perfect security is just as hard as perfect privacy.
Thank you! I hope that maybe one day the debate will get a proper resolution. I'm disappointed I couldn't be the one to provide that.
mox
in reply to The 8232 Project • • •The security provided by a browser is constantly changing, as the vulnerabilities, attacks, and countermeasures are constantly changing. It's a cat-and-mouse game that never ends.
The privacy provided by a browser would be difficult to measure, since it depends a lot on browsing habits, extensions, code changes between versions, etc.
There's no good way to calculate a metric for either type of protection, and even if there was, the metrics would be obsolete very quickly. For these reasons, I wouldn't have tried what you attempted here.
However, there is a very simple way to compare the major browsers on privacy and reach a pretty accurate conclusion: Compare the developers' incentives.
like this
DaGeek247 likes this.
Hirom
in reply to mox • • •It would be fair to compare browsers without adding extensions, with default settings.
This would show which browser have the best security and privacy out of the box. Also, the comparison would be practically impossible otherwise.
Most people use defaults, and I suspect a large portion of users install no extension, unless maybe if a tech-savy relative adds an adblocker.
0oWow
in reply to The 8232 Project • • •What matters to me is what tools the browser lets me use to complement it and harden it to my liking.
Chromium does not offer that. But if I’m going to use Chromium, it would be Brave browser, since it provides tools comparable to what I use in Firefox.
To me, how is a browser going to be attacked if the scripts the attacker would use are already blocked by my toolset? (Rhetorical)
disguised_doge
in reply to The 8232 Project • • •blurg
in reply to The 8232 Project • • •Ad-Free Privacy Tool/Service Recommendations - Privacy Guides
www.privacyguides.orgMimicJar
in reply to The 8232 Project • • •That's the only conclusion I would have trusted. Otherwise you should have been awarded the tech equivalent of a Nobel Prize.
Security (and privacy) is not a zero sum game. That isn't to say we shouldn't discuss it. That isn't to say we can't point out clear advantages.
In any case, I appreciate the write up.
TranquilTurbulence
in reply to The 8232 Project • • •Good job. This post shows that it’s s complicated topic, so squeezing it into a binary answer just isn’t going to work.
However, when it comers specific details, such as your data being sucked up to the servers of a creepy company, you can definitely provide clear answers. In situations like this, I tend to make a spreadsheet that lists all the useful details and rates each browser accordingly. Then, you give your subjective weight to each detail, and calculate a weighted average of each brewer. This final score is highly subjective and debatable, but at least you have some sort of answer that helps you decide what’s best for you.
far_university190
in reply to TranquilTurbulence • • •!privacy@lemmy.ml beer review when?
TranquilTurbulence
in reply to far_university190 • • •DollarColonial
in reply to The 8232 Project • • •As I would trust GrapheneOS for doing a insane job based on Android, if they make their "own" version of Chromium, I don't have much doubts that they know what they talking about on Chromium/Firefox.
Also looking at their forum, they answer lot of questions I can barely understand
The 8232 Project
in reply to DollarColonial • • •brrt
in reply to The 8232 Project • • •Thank you for the work you put into this.
A suggestion if you don’t mind: Use headings and bullet points to structure the text and make it easier for the reader (and maybe yourself) to take in the information. As it is it’s a wall of text that is hard to parse.
Eugenia
in reply to The 8232 Project • • •Actually, I'd trust the Graphene guys' evaluation. They do know what they're talking about there. And it's true that Playstore is more secure than foss store offerings, unfortunately. You see, these are built securely. Google is a security-driven company. That much is true, and I know that first hand. BUT they are not a PRIVACY-driven company. There is a difference here.
What we need, is a totally de-googled Chromium with added hardened extensions (e.g. bringing back the v2 manifest to run various privacy and security extensions). This would have more security than Firefox, but also more privacy.
I believe that's the best way forward, because creating a new web browser from scratch with these performance expectations, is a pipe dream (looking at you, ladybird). So, yeah, the open source community needs to fork chromium, not firefox. Firefox was never great to begin with as a technology, it's measurably slower than Chrome for example, and it uses a LOT more RAM. Linux users are known to want to resurrect old computers with less than 4 GB of ram (I'm one of them), firefox can't deliver that. I always have to resort to Chrome to make it bearable. But I rather use an official foss fork instead. One that is trusted.