how let run a LLM learning process driven by system ram and cpu model outsource to metis device?
or better say how to load 4,7GB into metis m.2 with 1GB
Page 1 / 1
Not sure you can. The Metis M.2 is designed specifically for inference, rather than training. That part still needs to run on a CPU or GPU.
As for running large models like a 4.7GB LLM, the SDK handles this by splitting the model and streaming the parts needed for each stage of inference. So the full model doesn’t have to live entirely on the Metis — it just processes chunks as required.
What is it you’re working on?
- frist to setup a raspberry pi 5 16GB with a metis device
- second DeepFaceLab for aarch64
- third run a LLM within this specs of hardware
the metis device can handle threads and graphics … sounds good
so is must be possible to run DeepFaceLAB within this device on a raspberry pi 5
the metis device can also handle threads (run in 1G of memory) … well with this specs must be possible
to do something with it
Reply
Sign up
Already have an account? Login
Log in, or create an Axelera AI account
Log In or Register HereEnter your E-mail address. We'll send you an e-mail with instructions to reset your password.