The Carbon Cost of Reasoning

2026-10-01T00:00:00+00:00

Full title</dt>

The Carbon Cost of Reasoning: Benchmarking Energy Efficiency in Fine-Tuning Gemma 4 for Mathematical Logic</dd>

Author</dt>

Mohammed Akram Khan Lodi</strong> (independent research, B.S. Abdur Rahman Crescent Institute of Science and Technology)</dd>

Presented</dt>

National AI Summit on Industry 5.0, April 2026, Chennai. Accepted & presented, Paper ID AIS 052</dd>

Status</dt>

Pipeline built · full experimental runs in progress</span></dd> </dl>

Proceedings</a></li>
Code: released with results</span></li> </ul>
TL;DR</h2>
Fine-tuning is the standard way to specialize a base model for multi-step reasoning, and its accuracy</em> is well documented. Its environmental</em> cost (energy drawn and CO₂ emitted) mostly goes unmeasured. Practitioners choose between Full Fine-Tuning, LoRA, and QLoRA</strong> based on accuracy and available compute, with no standard way to weigh what that choice costs.</p>
This project runs a controlled, reproducible comparison</strong> of the three methods on a small open-weight model for grade-school math reasoning, measuring accuracy and energy side by side. It introduces two tools for reading the result:</p>

the Green Gap</strong>: the point in training where each additional kWh stops buying meaningful accuracy; and</li>
the Reasoning Efficiency Index (REI)</strong>: accuracy gained over zero-shot, per kWh consumed.</li> </ul>
Research questions</h2>

How much energy and CO₂</strong> do Full Fine-Tuning, LoRA, and QLoRA each consume when specializing a model for multi-step mathematical reasoning?</li>
At what point does additional energy stop yielding meaningful accuracy gains</strong>, i.e. where is the Green Gap?</li>
Can a standardized metric make environmental efficiency as comparable across methods as accuracy already is</strong>?</li> </ol>
Methodology</h2>
</th> </th></tr></thead>

Base model</strong></td> Gemma 4 E2B-it (Google, ~5.1B stored parameters, Apache 2.0)</td></tr>
Benchmark</strong></td> GSM8K</a>: grade-school math word problems requiring chain-of-thought, scored by exact match on the final numeric answer</td></tr>
Methods</strong></td> Full Fine-Tuning · LoRA (ranks 8 / 16 / 32) · QLoRA (4-bit NF4, ranks 8 / 16 / 32)</td></tr>
Energy tracking</strong></td> CodeCarbon</a>: GPU, CPU and RAM energy (kWh) and estimated CO₂e per run, using the cloud region’s grid carbon intensity</td></tr> </tbody></table>
Full Fine-Tuning is backbone-only.</strong> The embedding and multimodal encoder layers are frozen (about 1.9B trainable parameters). This matches the architecture’s own pretraining conventions and</em> covers the same parts of the network that LoRA and QLoRA can reach, so the comparison is between methods rather than between different amounts of model being trained.</p>
Controls.</strong> Every condition uses the same fixed prompt template, dataset split, and evaluation protocol. Observed differences should come from the fine-tuning method, not from setup variables.</p>
Metrics</h2>
Reasoning Efficiency Index (REI)</h3>
Accuracy gained over the zero-shot baseline, per unit of energy:</p>
$$\text{REI} = \frac{\text{Acc}_{\text{fine-tuned}} - \text{Acc}_{\text{zero-shot}}}{E_{\text{training}}\ (\text{kWh})}$$</div>
REI turns “which method is greener?” into a single number that can be compared across methods, ranks, and, if adopted, across papers.</p>
The Green Gap</h3>
Accuracy checkpoints are logged against cumulative energy</strong> throughout training. That gives an accuracy-vs-kWh curve for each method. The Green Gap is the point on that curve where the marginal gain</p>
$$\frac{\Delta\,\text{Accuracy}}{\Delta\,\text{Energy (kWh)}}$$</div>
drops sharply. Past it, you’re mostly paying for energy, not reasoning. The rank ablations show how that point shifts with adapter capacity</strong>.</p>
Experimental design</h2>

Condition</th> Configuration</th> Repetitions</th> Role</th></tr></thead>

Full Fine-Tuning</td> Backbone only, bf16</td> 3</td> Core</td></tr>
LoRA</td> Rank 16, α = 32</td> 3</td> Core</td></tr>
QLoRA</td> 4-bit NF4, rank 16</td> 3</td> Core</td></tr>
LoRA</td> Ranks 8 & 32</td> 1 each</td> Ablation</td></tr>
QLoRA</td> Ranks 8 & 32</td> 1 each</td> Ablation</td></tr> </tbody></table> </div>
The 9 core runs</strong> support a statistically meaningful comparison across methods. The 4 ablation runs</strong> trace how the Green Gap moves with adapter rank. A zero-shot baseline evaluation anchors REI.</p>
Infrastructure</h2>
All billed fine-tuning, baseline evaluation, and energy-tracked runs happen on AWS EC2</strong> (us-east-1). Fixed-spec hardware with per-instance telemetry is what makes the energy numbers credible; managed fine-tuning APIs don’t expose hardware-level energy data.</p>

Instance</th> GPU</th> Used for</th> Why</th></tr></thead>

g5.xlarge</td> 1× NVIDIA A10G, 24 GB</td> Zero-shot baseline, LoRA, QLoRA</td> Enough memory for frozen/quantized-base training and batched inference</td></tr>
g6e.xlarge</td> 1× NVIDIA L40S, 48 GB</td> Full Fine-Tuning</td> ~1.9B trainable parameters need more headroom than the A10G’s 24 GB</td></tr> </tbody></table> </div>
To avoid wasting compute, all pipeline development (smoke tests, prompt and evaluation calibration) is done first on a free-tier Colab T4. The full study is budgeted at about $30–$50</strong>, with an AWS Budget alert at 50% and ~90% of a $50 cap, and instances run only for the duration of a specific experiment.</p>
Plan & status</h2>
Phase</th> Work</th> Status</th></tr></thead>

1. Pipeline validation</td> Smoke tests, prompt/eval calibration on Colab</td> Done</span></td></tr>
2. Baseline & core runs</td> Zero-shot baseline; Full FT / LoRA / QLoRA on AWS</td> In progress</span></td></tr>
3. Ablations & analysis</td> Rank ablations, Green Gap curves, REI</td> Planned</td></tr>
4. Write-up</td> Results synthesis, full paper</td> Planned</td></tr> </tbody></table>

The study design and motivation were presented at the National AI Summit on Industry 5.0 (April 2026). The full set of energy-tracked runs is still running, so no final numbers are reported here yet.</strong> The code, logs, and results will be released together.</p> </blockquote>
Expected contributions</h2>

A reproducible, open-source benchmark</strong> of the environmental cost of three widely used fine-tuning methods on a reasoning task.</li>
The Green Gap</strong> as a named, measurable phenomenon in fine-tuning cost–benefit analysis.</li>
The Reasoning Efficiency Index</strong> as a reusable metric others can apply to their own fine-tuning comparisons.</li> </ul>
Why this matters to me</h2>
My other project, Self-Harnessing RLMs</a>, improves a model without</em> touching its weights. This one asks what touching the weights actually costs. Together they’re two sides of one question I keep coming back to: where should the compute for better reasoning go, and is it worth it?</strong></p>

Akram - fine-tuning

The Carbon Cost of Reasoning