Effective Date: April 27, 2026 · Last Updated: April 27, 2026
DreamLab Solutions LLC built LLM Eval Suite as a macOS application for evaluating AI model outputs using LLM-as-judge methodology.
This app does not collect, store, or transmit any personal user data. All data — including evaluation runs, prompt variants, datasets, and configuration — is stored locally on your device at ~/Library/Application Support/LLMEvalSuite/.
When you configure a third party AI provider, such as OpenAI, Claude, or MiniMax, API key in Settings, the app makes API calls to the third party servers solely for generating and scoring evaluation outputs. These calls are governed by the third party's privacy policy. The app does not associate your API usage with any personal identifier beyond the API key itself.
Apple's on-device Foundation Models are used for local generation when available. No data leaves your device for generation.
This app does not include any third-party analytics, advertising, or tracking frameworks.
This app is not directed to users under the age of 13 and does not knowingly collect information from children.
If this privacy policy changes, the updated version will be posted alongside the app update.
If you have questions about this Privacy Policy or our privacy practices, contact us at:
Email: francisco@dreamlabsolutionsllc.com
Website: www.dreamlabsolutionsllc.com
