Mac Studio Apple Support me on patreon! jakkuh patreon apple offered to loan me four top of the line mac studio’s with their m3 ultra chip and a combined 1500gb of unified memory to try out exo. With exo, we've been running llms on clusters of apple mac studios with m3 ultra chips. the mac studio has 512gb of unified memory at 819 gb s, but the gpu only has ~26 tflops of fp16 performance.
Two Nvidia Dgx Spark Systems Fused With M3 Ultra Mac Studio To Deliver Apple's recent mac studio clustering breakthrough, enabled by rdma over thunderbolt and the public release of exo 1.0, allows running massive ai models locally with unprecedented speed and efficiency. Exo's newest demo combines two of nvidia's dgx spark systems with apple's m3 ultra–powered mac studio to make use of the disparate strengths of each machine: spark has more raw. A new demo project demonstrated how two nvidia dgx spark stations can work in tandem with apple mac studio on m3 chip ultra, creating a unified solution for accelerated model inference. Developers won't need any special hardware to build clusters, just standard thunderbolt 5 cables and compatible macs. in a demo, i watched as a cluster of four mac studios loaded and ran.
Mac Studio Mac Does That Apple Ca A new demo project demonstrated how two nvidia dgx spark stations can work in tandem with apple mac studio on m3 chip ultra, creating a unified solution for accelerated model inference. Developers won't need any special hardware to build clusters, just standard thunderbolt 5 cables and compatible macs. in a demo, i watched as a cluster of four mac studios loaded and ran. The goal of this experiment was to check if a cluster of five mac studios with m2 ultra chips and 64 gb of unified memory each could handle the task, leveraging apple’s unified memory architecture to compensate for the lack of dedicated vram. Even with unified memory and clustering, the macs couldn’t match an nvidia based ai server. the lack of tensor cores (which nvidia gpus use to accelerate ai computations) made a significant difference. They run prefill on the spark, streaming the kv cache to the mac over 10gb ethernet. they can start streaming earlier layers while the later layers are still being calculated. Exo is an open source software stack making use of apple’s mlx framework to make setting up this system easy. plug the devices into each other, open exo on each mac, download a model and it runs.
Comments are closed.