From an article entitled “Addressing any remaining LLM skepticism”
" But but but… what about hallucinations!?
It doesn’t matter. All issues with current LLMs can be easily overcome to the 98th percentile of accuracy with minor supplementation. Microsoft gets this. Anything you think is an issue with LLMs isn’t."
https://futureblind.com/p/addressing-any-remaining-llm-skepticism
The first 45 minutes of the most recent episode of The Vergecast is a deep dive into Arc Search and The Browser Company’s vision via an interview with its CEO, Josh Miller:
It is a really great interview. A few observations:
“It is kind of unbelievable how much of humanity’s time is wasted on how atrocious the mobile web is in support of the search ad business model … and we’ve gotta find out a better way to get content creators and publishers paid, because it’s not gonna work, and it’s not been working — it’s not been working since Craiglist.”
It looks like an extension of the same vision that drives the ChromeBook, and even though Arc is developing that idea out in innovative and useful ways, it’s still bound by the same core limitations. ChromeOS has carved out a piece of the market, but it hasn’t driven Windows and macOS or the local applications that run on them out of the mainstream.
BTW, there’s something kind of Apple-ly about Arc. It probably won’t happen, but I wouldn’t be completely surprised if Apple bought it, swapped out Chromium for WebKit, and incorporated it into the next Safari. It is written in Swift, after all.
The biggest downside of that would be the limitations of Apple’s extensions support.
Yes. This feels very much like a certain group of politicians. Thoughts and prayers. (To be clear gun violence is far bigger problem than LLMs wiping out some parts of the web).
I think any on-device solution will be severely limited. All but the most limited LLMs need to run in datacenters, and my digital assistant will need access to my data as well as the internet. How will “iPhone Siri” know what I ask “HomePod Siri” about concerning some files on “MacBook Siri”? IMO, both our digital assistant and our data may need to reside in the cloud.
I can’t wait to listen – thanks for the recommendation. I really like where Arc Search is going, to be honest. I can see that “something” needs to be done to ensure creators get more credit/traffic for their work. But Arc Search already does more than many LLMs in that they put the source links into the output, which I really like.
And for those concerned, I absolutely click through to relevant links of interest to dive deeper.
Very true - on-device models are puny little side projects right now, especially if we consider something like RAG to be the path towards creating verified text without complete bullshit.
But it’s early days. I can run quite large models with Ollama locally, and that’s with chips designed before the true LLM breakthrough in october 2022.
Apple are rumored to be releasing their own on-device models this year, presumably at WWDC - and they will be created specifically FOR my chip, which means they should be even faster.
Then consider the chips Apple are designing right now! Perhaps it is too late to make changes in the M4, but the M5 could be a LLM powerhouse!
The future’s so bright, we’ve got to wear shades!
True, it’s early days. Apple currently supports iPhones for at least 5 years. How many of the 1.9 billion iPhone users would be in a hurry to upgrade to an M5 if someone offered a solution that worked on their existing smartphone? iPhone users already upgrade less often than Android users.
To be fair to Miller, my interpretation is that he is abhorred that your upstanding ad-free business has to get found through the wash of ads and SEO in order to find a customer. E.g., I imagine that if someone searches Google for “mark levison scrum problems”* they might find other businesses before yours, which is pretty nefarious.
But yes, them destroying those conventional channels and then saying “guess everyone else will have to rethink everything!” is kind of rough.
And … it is worth anticipating that these Arc Search Browse For Me pages could certainly have nice-looking ads embedded at the top in ten years when The Browser Company’s investors decide their 10x payback is due.
*: If you’re havin’ scrum problems, I feel bad for you, son / I got ninety-nine problems but a sprint ain’t one
Do you really think they will be around in ten years? If the ideal is successful I would expect an Apple, Google, Microsoft, etc. deep pockets company would either buy them, or create a similar product that would quickly overshadow them.
I’m doing a lot of LLM work recently with an MBP. Current LLM’s need RAM, then GPU. Models Llama - 72B - are trying to predict everything. If you don’t need to chat about all of human knowledge then you find ways to cut down the model. Another trick is around context length if you need to predict only a paragraph at a time, you can use less RAM.
In addition, there will be far more interesting uses of these tools than spiting out your next email or summarizing a thread on MPU.
I think there is a lot of room for Apple do on device with existing devices going back a few years.
(An aside - I think chips are generally worked on 3+ yrs head. So a genius idea today is only going appear in M6/M7 generation).
I had a short-lived experience.
I always wear a tinfoil hat when I use Arc, so it’s safe.
I used to agree with this, still do if it’s some random app I downloaded on my iPhone (not that I do that anymore). But right now I am trying to find a browser on Windows I like. Some of them don’t have accounts and I have to jump through hoops to get them to remember settings across computers. Brave I had to type in a set of 25 random words to sync. At that point I just wanted to create an account (you can’t).
Point is, sometimes it is a good thing. And Arc is way past the point of some random browser that might be good. It’s worth account creation.
On the other hand I have been a Mac user of the app for well over a year and I cannot get into the Windows beta for some reason. It is extremely annoying. And the lack of iPad apps.
I am all for Arc, specially their attention to little details that make your browsing life easier. Arc Search seems very interesting, and they are putting on their disrupter’s hat, changing the web and whatnot.
But I will not be fooled to think that as soon as they see a solid profit pool they will not trade with my browsing data. These browsers that set up config sync schemes on their own servers (“create an account to sync”, “create an account to use our powerful AI assistant”) are easily building a data collection engine that is comparable to the one that Google build in decades.
The key for Arc is how fast Google can move, and I think Google is now running full steam (new Bard runninng Gemini, for example). If Google can activate all the data it knows about me and use it to give me relevant AI answers…
Arc doesn’t collect browsing data. They do collect product usage data (using the quick launcher, having pinned tabs, that you split tabs etc.) Tab syncing is either in iCloud or their own e2ee service. They see your content from the hosted services they provide (creating an arc.net share link and creating an easel); those are optional.
The AI queries go straight to OpenAI and Anthropic; neither company retains the contents.
Arc would have to change their privacy policy and software to make money off us that way.
And yes, Google should be successful at this concept, if they choose to try to cannibalize search.
It appears that option is on the table.
https://www.wired.com/story/google-prepares-for-a-future-where-search-isnt-king/
PSA: Arc announced today that iCloud sync is going away, and their own sync service will be used going forward.
That seemed inevitable once they started building a version for a non-Apple OS. Idk of any cross-platform browsers that sync via iCloud.
You read too much into it.
The problem to me is the mandatory account creation in order to use the browser.
I can use all other browsers just fine without an account.
The account is optional, to sync bookmarks, settings etc.