Category: Apple

gpt4all on an m1 macbook pro

whbeebe

I’ve been tinkering a bit with large language models small enough to run locally on my M1 MacBook Pro without having to reach out across the internets. Local LLMs means not having to send anything across that might be collected, and thus losing control on whatever I type in (not that whatever I type in is worth collecting…).

You can go look at GPT4All here: https://gpt4all.io/index.html

The application, when first installed, can’t do anything. You need to begin to add LLMs via a curated selection it presents. These are the eleven currently listed.

  1. Wizard V1.1
  2. GPT4All Falcon (the one I’m currently using)
  3. Hermes
  4. ChatGPT-3.5 Turbo
  5. ChatGPT-4
  6. Snoozy
  7. Mini Orca
  8. Mini Orca (small)
  9. Mini Orca (large)
  10. Wizard uncensorted
  11. Llama-2-7B Chat

I’ve barely scratched the surface, having just installed the application and then selected a given model. My criterial right now for selecting a model is that there is no link back to the cloud and that it can run in the “limited” 16 GiB of memory on my MacBook. GPT4All Falcon, for example, will run in 8 GiB.

If you stop and think about the resources required by Falcon it suddenly becomes apparent that it could run on an Apple iPhone if the iPhone had enough memory. Today’s contemporary iPhones certainly have the processor horsepower and more than enough local storage to hold such a model. The problem is that the iPhone 15, Apple’s latest device, doesn’t. Perhaps the iPad Pros running with either the M1 or M2 has enough, but I know the iPhones do not.

Why would I want that much memory? To run one of the LLMs locally. And why do that? So that the LLM could “learn” about me from all the data that passes through my iPhone from me, and help the iPhone to be an even better aid. Smartphones, especially the iPhone, has long since passed the point of “good enough” to perform such tasks. The big question is how much power will that require, because there are always trade offs with portable computers. The more hardware you have on an iPhone, such as more memory, the more power the device will consume. The more processing the CPUs on the iPhone needs to perform, the more power the device will consume. As the saying goes, be careful what you ask for.

And there’s another reason for looking at this for local processing, and that’s in support of the intelligent home. My wife and I grow progressively older. We’re both in our early 70s. I would like a home with sensors and microphones and speakers to be able to assist our living here, rather than have to go live in a classical assisted living facility. And I do NOT want devices that require cloud connectivity and a monthly fee. Right now I have an intelligent [sic] doorbell which is absolute shit. I want far better, and I want local autonomy.

I haven’t even scratched the surface with any of this software. But it’s now approachable and I can run it locally on my MacBook, which I find absolutely amazing. I hope I can do more with it going forward.

faint praise for my macbook pro

whbeebe
I own the space gray version to the right.

It should be no secret how much of a critic I’ve become of Apple. I’ve long been critical of the price you pay for their technology, the lack of quality in their software, the opaqueness of their technology, and the way they determine when a given Apple device is obsolete and thus no longer eligible for macOS updates. However…

I have to admit that from time to time Apple will release a certain product that is worth the price of admission, even if I have to hold my nose (as it were) while I use it. That product is the one I’m writing this post on, Apple’s 2020 13″ M1 MacBook Pro.

I didn’t purchase this notebook when it was released, but December 2021 when it was being heavily discounted. The one I purchased has 16GB of memory and a 1TB SSD. At the time of purchase it was actually cheaper than an M1 iPad Pro and the Apple Magic Keyboard, even with more memory and storage. Especially with 1TB of storage, which made no sense as the iPad Pro M1 was the same SoC as in the MacBook Pro. I suspect that the 13″ version of the MacBook Pro didn’t fly off the shelves the way they had hoped, so they went on sale to clear the stock.

I’ve mentioned it before, but it bears repeating: this M1 MacBook Pro is every bit as performant as the one-year-older 16″ MacBook Pro with the Intel hex core i9.  Combined with the M1’s incredibly low power usage, and you have a formidable competitor to anything running the Intel ISA back then or even now, three years on. All the hype and praise heaped on the M1 is real.

This M1 MacBook is the first notebook where I could leave the power brick back at home/office without any qualms, and certainly one that didn’t require I find an AC power plug to plug in my notebook’s charger when I did carry it, just so I could be sure I didn’t run out of juice. I’ve worked on this MacBook for days at a time before I even began to think about plugging it back in to charge the battery.

This MacBook also has the strip across the top of the keyboard instead of the function keys. I consider that change a bit of engineering brilliance, but I’m alone in that judgement. Apple abandoned the strip in favor of putting the function keys back because too many Apple stans complained. Yet another example of why we can’t have nice things.

Since I brought up the Apple Magic Keyboard earlier, let me also state that I find the MacBook Pro’s keyboard nearly perfect, and far better (again, in my opinion) than the feel of the Apple Magic Keyboard (or at least the first version I tried). I judge this MacBook Pro the “Mary Poppins” of MacBook Pros, a device practically perfect in every way. As for overall construction, the aluminum body and solid hinge make for a very stable platform that can be used anywhere. There is no wobble, no movement whatsoever between the screen and the keyboard.

Do I recommend this? Since we’re now well into the third year after it’s initial release, I give a qualified recommendation. If you can find a new-in-the-box version for under US$1,000, I’d say you couldn’t go wrong. If you can find a more current version using the M2 that’s not much more than US$1,000, then perhaps yes.

While there is a performance increase between the M1 and the M2, I’m not so sure you could tell the difference in day-to-day work. Furthermore, when Apple released the faster M2 machines they started using cost cutting designs that did have a negative impact on performance. In particular, Apple started to use a single unified SSD chip for storage on machines with 256GB storage or less. Thus, if you want the advantage of the SoC’s wide bus you’d need to buy an M2 MacBook Pro with 512GB or greater to avoid that performance bottleneck. Since my machine has 1TB, and if I were in the market to upgrade (which I am most definitely not) then I wouldn’t run into this issue. If you’re looking for bargains you won’t find them at an Apple store. You’re better off shopping places like B&H Photo Video, Adorama, and Amazon. For example I purchased mine at B&H Photo Video.

At some point I’m sure Apple with make my M1 MacBook Pro a “legacy” device and thus not able to run the latest macOS. I just hope when that day comes that Asahi Linux (or some other distribution) is ready enough I can drop it onto this machine and keep on going.