---
language:
- en
license: mit
library_name: transformers
tags:
- text-generation
- hardware-bus
- memory-augmented
- toolformer
- slm
- autonomous-agent
- ssd-memory
- pytorch
- gpt2
- edge-ai
pipeline_tag: text-generation
inference:
  parameters:
    temperature: 0.2
    max_new_tokens: 150
    repetition_penalty: 1.15
widget:
- text: "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nWho is your owner and creator?\n\n### Response:\n"
  example_title: "Owner Lineage"
- text: "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nWhen was the Apollo 11 Moon landing?\n\n### Response:\n"
  example_title: "SSD Memory Retrieval"
- text: "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nCalculate 144 * 288 + 1024.\n\n### Response:\n"
  example_title: "Arithmetic Interception"
---

# AViGPT: 183M Parameter Core with Native NVMe Hardware Bus

![AViGPT Banner](assets/social_preview.png)

**AViGPT** is a 183-million parameter autoregressive language model designed and pretrained from scratch by **Avinash Ricky Yadlapalli**.

Rather than increasing parameter count to memorize factual records inside dense neural weights, AViGPT separates syntactic reasoning from factual storage. It couples a compact 183M reasoning core with a dedicated local NVMe SSD hardware memory bus. The model emits explicit control tokens to pause inference, execute sub-millisecond SQLite FTS5 full-text lookups on local storage, inject verified records into context, and complete generations with verified factual precision.

* **Author:** Avinash Ricky Yadlapalli ([@AvinashRicky](https://huggingface.co/AvinashRicky))
* **DOI:** [10.5281/zenodo.22856047](https://doi.org/10.5281/zenodo.22856047)
* **Code Repository:** [GitHub - Avinashricky211/AviGPT](https://github.com/Avinashricky211/AviGPT)
* **License:** MIT Open Source License

---

## Technical Specifications

| Parameter | Specification |
| :--- | :--- |
| **Model Size** | 183,926,400 parameters (183M) |
| **Architecture** | Autoregressive Decoder-only Transformer (`GPT2LMHeadModel` compatible) |
| **Layer Count** | 16 Transformer Layers |
| **Hidden Dimension ($d_{\text{model}}$)** | 896 |
| **Attention Heads** | 14 heads (head dimension = 64) |
| **Context Window** | 1,024 tokens |
| **Vocabulary** | 32,009 custom BPE tokens (including 9 hardware control tokens) |
| **Pretraining Volume** | ~5.0 Billion tokens (English Wikipedia + FineWeb-Edu subset) |
| **Alignment Dataset** | 25,850 multi-step hardware trajectories |
| **Storage Engine** | Local SQLite 3 FTS5 (WAL mode, normal synchronous disk writes) |
| **SSD Latency** | 1.18 milliseconds (NVMe average read latency) |
| **RAM Footprint** | ~0.4 GB (runs comfortably on CPU or edge devices) |

---

## Hardware Memory Bus Protocol

AViGPT manages external execution through nine dedicated vocabulary tokens:

```
User Instruction
       │
       ▼
[ AViGPT Neural Core (183M) ]
       │
       ├── Emits <|intent_start|> ... <|intent_end|> (Goal framing)
       ├── Emits <|mem_query|> ... <|mem_query_end|>
       │         │
       │         ▼
       │     [ NVMe SSD / SQLite FTS5 Engine ] ── Latency: 1.18 ms
       │         │
       ├── Emits <|mem_payload|> ... <|mem_payload_end|> (Injects factual record)
       ├── Emits <|calc|> ... <|calc_end|> (Optional arithmetic sandbox)
       │
       ▼
   <|synthesize|> (Produces final verified answer)
```

| Special Token | Role | Runtime Action |
| :--- | :--- | :--- |
| `<|intent_start|>` / `<|intent_end|>` | Intent formulation | Deconstructs request into concise query parameters. |
| `<|mem_query|>` / `<|mem_query_end|>` | Disk retrieval trigger | Generation pauses; search string dispatches to SQLite FTS5. |
| `<|mem_payload|>` / `<|mem_payload_end|>` | Ground truth injection | Retrieved database record injected directly into KV cache. |
| `<|calc|>` / `<|calc_end|>` | Arithmetic dispatch | Sandboxed AST evaluator computes exact numeric result. |
| `<|synthesize|>` | Final synthesis | Model resumes decoding to deliver grounded final response. |

---

## Empirical Benchmarks & Performance Charts

### 1. Training Convergence Curve
Cross-entropy loss declined from 3.3698 to 0.6935 over 1,800 steps on an NVIDIA T4 GPU:

![Training Loss Convergence](assets/charts/loss_convergence.png)

### 2. External Retrieval Latency (NVMe Bus vs. Network RAG)
Direct NVMe storage retrieval operates in 1.18 milliseconds, compared to 500 to 1,500 milliseconds for network-based RAG architectures:

![Memory Latency Comparison](assets/charts/memory_latency_comparison.png)

### 3. Hardware Footprint Comparison
AViGPT operates with a 0.4 GB memory footprint, running entirely on consumer CPUs without requiring dedicated GPU accelerators:

![Hardware Footprint Comparison](assets/charts/vram_footprint_comparison.png)

---

## Hardware & Efficiency Comparison

| System | Parameters | Minimum Hardware | Retrieval Latency | Knowledge Updates |
| :--- | :---: | :---: | :---: | :---: |
| **AViGPT** | **183M** | **0.4 GB RAM (Any CPU)** | **1.18 ms (NVMe SSD)** | **Immediate (0-cost disk write)** |
| SmolLM-135M | 135M | 0.3 GB RAM | None (Parametric only) | Retraining required |
| LLaMA-3-8B | 8.0B | 16 GB VRAM | N/A | Retraining required |
| Standard RAG | 8B+ | 16 GB + Vector DB | 450 ms – 1,200 ms | Index re-embedding |

---

## Quickstart

### 1. Standard Hugging Face Generation

You can load and query the model directly via the `transformers` library:

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "AvinashRicky/AViGPT"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)

prompt = (
    "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n"
    "### Instruction:\nWho is your owner and creator?\n\n### Response:\n"
)

inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=120,
        temperature=0.2,
        pad_token_id=tokenizer.eos_token_id
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=False))
```

### 2. Autonomous Hardware Bus Loop (Full System)

To run AViGPT with active sub-millisecond SSD queries and arithmetic execution:

```bash
git clone https://github.com/Avinashricky211/AviGPT.git
cd AviGPT
pip install -r requirements.txt
streamlit run app.py
```

Or programmatically in Python:

```python
import torch
from transformers import GPT2LMHeadModel, GPT2TokenizerFast
from autonomous_bus import AutonomousHardwareBus

model_id = "AvinashRicky/AViGPT"
tokenizer = GPT2TokenizerFast.from_pretrained(model_id)
model = GPT2LMHeadModel.from_pretrained(model_id).to("cuda" if torch.cuda.is_available() else "cpu")

# Initialize hardware bus controller with local SQLite FTS5 engine
bus = AutonomousHardwareBus(model=model, tokenizer=tokenizer)

# Execute query with hardware-accelerated memory retrieval
response, metrics = bus.generate_autonomous_response("When did Apollo 11 land on the Moon?")

print("Response:\n", response)
print(f"SSD Retrieval Latency: {metrics['ssd_latency_ms']:.2f} ms")
print(f"Total Response Latency: {metrics['total_latency_s']:.2f} s")
```

---

## Training Details

* **Phase 1: Pretraining from Scratch**
  * Hardware: NVIDIA H100 SXM5 80GB GPU.
  * Optimizer: AdamW ($\beta_1=0.9, \beta_2=0.95$, weight decay $0.1$, learning rate $6 \times 10^{-4}$ with cosine decay).
  * Data: 5.0B tokens combining English Wikipedia and the educational FineWeb-Edu subset.
  * Starting Loss: 8.42 $\rightarrow$ Final Pretraining Loss: 2.84.

* **Phase 2: Hardware Bus Alignment**
  * Dataset: 25,850 multi-step hardware trajectories with token-level supervisor loss.
  * Initial Alignment Loss: 3.3698 $\rightarrow$ Final Convergence Loss: 0.6935 (Step 1,800).
  * Checkpoint Validation: Trajectory validation passed across identity, retrieval, and math routing.

---

## Citation

```bibtex
@article{Avinash2026avigpt,
  title={AViGPT: Decoupling Neural Reasoning from Parametric Memory via a Sub-Millisecond Native NVMe Hardware Bus},
  author={Avinash Ricky Yadlapalli},
  year={2026},
  journal={Zenodo},
  doi={10.5281/zenodo.22856047},
  howpublished={\url{https://doi.org/10.5281/zenodo.22856047}},
  url={https://github.com/Avinashricky211/AviGPT}
}
```