GGML is an open-source, high-performance matrix library designed for machine learning and other applications requiring matrix operations. It stands out for its lightweight nature, simplicity, and focus on supporting a wide range of platforms, including CPUs, GPUs, and specialized AI accelerators.
Assume you have a file named ggml-medium-350m-q4_0.bin. Here is the workflow. ggmlmediumbin work
./main -m llama-2-13b.q4_0.bin -p "Explain quantum computing" -n 100