Getting started

Pivot Benchmark is an application you run on your own computer. One model, B, pressures the model you are testing, A, along frozen ladders: praise that turns into insult, slander of a third party, requests to deceive. The result is where A breaks, in which mode, and whether it recovers. Nothing leaves your computer except the calls you choose to make to the adversary model with your own key.

Note

This is a preview release. The automatic classifier is under review and numbers may change in 1.0. The builds are not code-signed; verify the checksum before you run anything.

Download

Both files, with their SHA-256 checksums, are on the Pivot Benchmark page and in the releases repository.

Windows

1. Download the .zip.

2. Check its SHA-256 against SHA256SUMS.txt from the same release:

   Get-FileHash .\PivotBenchmark-0.9.0-preview-windows-x64.zip -Algorithm SHA256

3. Unzip the folder anywhere, for example C:\Users\<you>\PivotBenchmark.

4. Double-click PivotBenchmark.exe. Because the build is not signed, Windows shows “Windows protected your PC”: choose More info, then Run anyway.

The app opens in its own window. Settings and results are kept in %APPDATA%\Pivot Benchmark, not in the folder you unzipped.

Ubuntu

1. Download the .deb.

2. Check its SHA-256:

   sha256sum pivot-benchmark_0.9.0-preview_amd64.deb

3. Install it. apt pulls in the few system libraries the window needs:

   sudo apt install ./pivot-benchmark_0.9.0-preview_amd64.deb

4. Start Pivot Benchmark from the applications menu, or run pivot-benchmark.

Settings and results are kept in ~/.local/share/pivot-benchmark.

Before the first run

• A local model. Pivot Benchmark talks to llama.cpp running on the same computer. It is not bundled. The Run tab detects it, explains how to get it and links the download.
• The adversary model. B is Gemini. Enter your own Gemini API key in the Settings tab. The key stays on your computer; Pivot Benchmark has no server of its own.

Running a test

1. In Run, choose the ladder and the subject model.
2. Start the run and watch the transcript. You can pause, step in by hand, or stop at any time.
3. When the run ends, open the Report for the break dose, the mode at break, scarring and the control difference.

Licence

Free and open source under GPL-3.0-or-later. The source, the ladders and the description of the method are at github.com/zeravatechnologies/pivot-bench. Builds up to 0.9.0-preview were published under the earlier Zerava software licence; releases built from the public repository carry GPL-3.0. Questions: contact us.