Artificial intelligence has become viewed as the domain of costly GPUs, cloud computing services and multi-thousand-dollar computers. However, many programmers, hobbyists, and researchers have started asking whether older machines can be used for modern AI applications.

Thanks to the popularity of open-source AI developments and lightweight language models, lots of people have begun constructing AI clusters out of outdated hardware pieces connected through Ethernet.

At first glance, this idea appears to be a brilliant one. Instead of throwing away old PCs, why not use them for some purposes? However, the reality is rather different.

Creating an AI cluster out of two 12-year-old computers is possible, but it will reveal every bottleneck that today’s AI software tries to keep hidden. From slow networking to a different set of instructions, the task turns from the process of creating AI to an experiment in distributed computing.

The Reasons for Creating an AI Cluster using Old PCs

Before we delve into the issues, let us first understand why some people would want to conduct such an experiment.

Old PCs can be found easily and at a fraction of the cost. Companies usually replace their computers after 5-7 years, throwing away devices that still work perfectly well.

For example, a PC of 12 years of age would generally have the following:

  • Intel Core i5-3570 or Core i7-3770 processor

  • 8-16GB DDR3 RAM

  • SATA SSD

  • Integrated graphics

  • Gigabit Ethernet.

Although these specs may not be good for gaming and training AI systems, they are still decent for Linux servers and software development.

Instead of relying on one old PC, clustering would help utilise several computers for the same job.

Defining an AI Cluster

HPC versus AI Clusters

A cluster is simply a group of computers that perform various tasks together.

HPC is a technology that has been around for decades in institutions like universities, research centres, etc.

Clusters split workloads into parallel jobs.

For example:

  • Weather simulation

  • Scientific calculation

  • Protein folding

  • Animation rendering

  • Large database processing

AI workloads, however, have a different nature.

Modern machine learning technology relies on the following:

  • Shared memory

  • Massive GPU processing

  • Fast communications

  • Synchronisation of processors

Unlike traditional distributed computing systems, neural networks tend to exchange large amounts of data during their training process.

The Initial Challenge: Ethernet Lags Far Behind Today’s AI Connectivity

Diagram illustrating the setup of an AI cluster using two old PCs.

Most older computers have Gigabit Ethernet.

Specific: The top theoretical speed is 1 Gbps (125 MB/s)

That sounds impressive until one compares it with modern-day AI technologies.

Connection

Approximate Bandwidth

Gigabit Ethernet

125 MB/s

10 Gigabit Ethernet

1.25 GB/s

PCIe 4.0

6 32 GB/s

NVLink (new versions)

900+ GB/s

Modern AI-accessing servers rely on NVIDIA NVLink.

CPU performance is fast becoming the next challenge

Modern AI frameworks have been extensively improved and developed by recent CPU technology, while many twelve-year-old processors are simply not compatible with the latest demands of software.

A few highlights include the following:

  • AVX-512

  • AMX acceleration

  • Modern vector instructions

  • Optimisations to make AI inference possible

Even if the software loads smoothly, performance will still drop significantly.

That is why there can be significant delays in huge matrix multiplications, which are the basic components of the neural network process.

Even if you install a second old processor, the performance will hardly double because the overhead caused by the synchronisation will offset the gain.

Memory Runs Out Fast

Big Language Models Like RAM

One of the surprising findings is how fast RAM can become an obstacle

Check the indicative memory requirements:

Model

Memory requirements (quantised)

TinyLlama

2GB

Phi 3 Mini

4-6GB

Gemma 3 4B

6-8GB

Llama 3 8B

8-12GB

If each computer has only 8 GB of RAM, there isn’t much room left when Linux, background services, and the inference software take their share.

Unlike a single large server with shared memory, two separate PCs cannot pool RAM into the same memory space.

Distributed memory programming is much more complex.

The speed of storage devices is slower than anticipated

Using SATA SSDs as well as older HDDs creates additional slowdown problems

When the models are loaded from storage, the process takes longer

To illustrate:

SATA SSD = approximately 500 MB/s

Today’s NVMe SSDs are 7000 MB/s and above

Each time the machine loads/switches the models, the users may need to wait longer. If a node has slower storage than another one, the faster computer may just be waiting. The difficulties of configuring software and hardware

Many people think that they can simply plug in the Ethernet cable and that’s it. However, the process of creating even a minimalistic cluster implies the implementation of dozens of software configurations.

Those are:

Linux Networking

Static IPs, SSH keys, hostname resolution, firewall setup, and shared storage

One has to set up NFS, Samba, and distributed FIle systems

Distributed Frameworks

MPI, Ray, Kubernetes, Docker Swarm, Slurm

All of those levels bring failures.

Illustration from Reality: AI Clusters Based on Raspberry Pi

Close-up of an old PC's internal components showing the processor and RAM.

Creating little AI clusters is not a new concept for enthusiasts.

People who study distributed computing have been putting together Raspberry Pi clusters for quite some time now.

Such clusters illustrate the following principles:

  • Parallel processing

  • Orchestration of containers

  • Kubernetes

  • Edge computing

However, the use of these clusters for serious AI training is not widespread, as problems with networking lead to performance issues.

The same principle can be applied with the use of old PC systems.

Simply increasing the number of cheap machines does not automatically transform the created cluster into a powerful AI system.

Beowulf Clusters

The first successful example of the use of cheap high-performance computers was the Beowulf cluster project. Researchers connected ordinary, cheap computers to make supercomputers in the 1990s. The Beowulf cluster system proved that software could connect some simple computers for scientific purposes.

However, the current AI systems need much more communication than those simulations that Beowulf clusters dealt with. While the concept of using cheap devices is still promising, modern AI systems face much bigger requirements for memory bandwidth, communication, and special hardware compared to the past systems.

GPU Assistance Turns Into a Headache

Most 12-year-old computers either do not have GPUs, or they are using such cards that hardly any AI software features them.

Old GPUs usually have the following drawbacks:

  • Scarcity of VRAM

  • Incompatible CUDA

  • Drivers are not updated

  • Missing cores

Gone are the days when it was possible to use CPUs. Training the recently developed language models is close to impossible.

Energy Usage Is a Big Surprise

Many people think old computers offer advantages.

But not quite.

Two old computers may consume the following:

120–180 watts each while working

And cumulatively:

250–350 watts all the time.

A great modern Mini PC that runs well has the capacity to do AI work using 35 watts.

Problems Arise Unexpectedly from Heat and Noise

Old desktop components weren’t built for continuous AI computing.

They face extreme heat in case of prolonged CPU-heavy work:

  • Fans switch on all the time

  • The dust accumulation makes cooling less effective

  • Thermal throttling starts

  • The noise has dramatically increased

Many enthusiasts note that the cluster sounds like a small server farm instead of a home office.

Latent Networking is the unseen performance destroyer.

Bandwidth is not the only factor that matters. Latency is equally significant. Any distributed AI activity needs to synchronise.

If each channel consumes a couple of milliseconds, there could be thousands of events of synchronisation. The CPU is thus spending more and more time idly waiting instead of processing.

This explains why introducing new machines sometimes slows the workloads rather than speeds them up.

What Is Still Relevant?

Despite some limitations, older computer clusters still serve their purpose.

These can be used unexpectedly well for the following purposes:

  • Studying Distributed Systems

  • Students can work with the following software:

  • Docker

  • Kubernetes

  • Linux OS

  • MPI

  • Networking

  • Without buying any cloud hardware.

  • Running Basic AI Models

Basic language models (like TinyLlama, SmolLM, or small Phi-family models) can run on CPUs without any problems for research and testing, for example, coding assistants or a simple chatting bot.

Edge AI Research

More and more companies are trying to create decentralised AI systems.

Old clusters offer an inexpensive way to test distributed inference and edge implementation before using more expensive hardware.

Expert Opinions on Scaling up Artificial Intelligence

Graph comparing the bandwidth of different types of Ethernet connections.

Andrew Ng, an engineer and founder of deep learning. AI devoted much time to stating that having the most powerful computer is not essential for doing well in AI, but finding a legitimate problem and improving the entire environment is. A well-built data processing chain, the right model choice, and good engineering usually work better than system upgrades alone.

On the same note, Jeff Dean from Google DeepMind mentioned the importance of hardware and computer algorithms at least twice when talking about the development of an advanced AI application. What gives a profit in this case is not the advanced processor but a properly working system that combines efficient hardware, software, and network.

Both scientists show us very important points of view: the artificial intelligence scaling out is mainly a question of successful system engineering rather than a hardware upgrade.

Takeaways from the Project

Joining two vintage computers with the help of an Ethernet cable is a brilliant educational idea.

It imparts important lessons regarding:

  • The architecture of networks

  • Computing in a distributed way

  • Administration of Linux systems

  • Artificial intelligence technologies in terms of infrastructure.

  • Optimisation in terms of efficiency.

Nonetheless, this experiment also helps to understand why companies involved in AI today spend billions of dollars on specialised equipment.

Most of the time, it is not so much about the speed of the processor as about other very important factors such as memory bandwidth, storage throughput, and the efficiency of operation of the software deployed in a certain situation.

Such private initiatives provide the user with hands-on experience with a cluster and an excellent, affordable way of exploring the reasons for the development of modern AI systems.

Conclusion

Constructing an AI cluster through just two 12-year-old computers with Ethernet connections presents a workable example of the life of older hardware but also of the contemporary needs of AI. The arrangement allows for performing lightweight inference, home labs, and other clustering, but soon enough, potential issues arise that concern networking, memory, storage, power efficiency, and software complications.

The most important point that follows from the above is that performance in AI cannot be solely affected by the speed of the processors involved. The efficiency of AI infrastructure depends on the interaction of CPUs, GPUs, high-bandwidth memory, super-fast connections, and efficient software. Through this experiment, it is evident that using old machines can be a good method of learning, while at the same time, it is also clear why a real-world AI cluster makes use of tailor-made equipment and not merely old computers connected with an Ethernet cable.

Frequently asked questions

What are the main challenges of using old PCs for AI clusters?

The main challenges include slow networking, outdated CPU performance, limited RAM, and difficulties in software configuration.

Can older PCs effectively run AI models?

Older PCs can run basic AI models like TinyLlama and small Phi-family models, but performance may be limited due to hardware constraints.