NIL-3B releases GGUF quantized weights, enabling local inference
Understand this story
The NIL-3B project has released the quantized model file nil-3b-q4_k_m.gguf, which can run locally in GGUF-compatible inference runtimes and start the chat script via python -m src.chat. The repository also provides upload_hf.py, used to push the GGUF file to a Hugging Face model repository. The project description states that the LICENSE in the root directory covers only the source code and does not independently confirm the licensing and redistribution rights for the model weights and training data; the project recommends confirming these rights before making the repository public, and documenting the model's base model, training data, and supported capabilities.
AI overview based on source reports · Updated 23 hr ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AINil-3B Open Source Sovereign AI
This repository hosts the nil-3b-q4_k_m.gguf file for running the NIL-3B project locally with a GGUF-compatible inference runtime, via a chat script that expects the file in the project root. It also documents an uploader script that uses a Hugging Face token and repo ID from a .env file to create the model repository if needed and upload the GGUF file. The root LICENSE covers only the project source code and does not by itself confirm license or redistribution rights for the model weights or training data.
Activity for this story
- Current comparable activity
- 5
- Peak comparable activity
- 10Nov 14 00:00
- Change in the past 24 hours
- –
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.