Testing Prompts

Endprompt includes a powerful test runner that lets you validate prompts before deploying to production. Test with single requests, bulk CSV uploads, or saved test datasets.

The Test Runner

Every prompt has a Test button that opens the test runner:

Auto-generated form based on your input schema
Real-time execution against the actual LLM
Response preview with timing and token usage
History of past test runs

Single Request Testing

Open the Test Runner

Click Test next to any prompt in your endpoint.

Fill in the Form

The form is generated from your input schema. Required fields are marked. Image fields display a file upload control (drag & drop or click to browse).

Select Options

Bypass Cache: Force a fresh LLM call
Model Override: Test with a different model

Run Test

Click Run Test or press Ctrl/Cmd + Enter.

Review Response

See the JSON response, latency, and token usage.

Test Results Panel

After running a test, you’ll see:

Section	Description
Response	The JSON output from the LLM
Latency	Time taken for the request
Tokens	Input and output token counts
Cost	Estimated cost of the request
Raw Output	Unprocessed LLM response

Toggle between Formatted and Raw views to debug JSON parsing issues.

Bulk CSV Testing

Test multiple inputs at once by uploading a CSV file:

Prepare Your CSV

Create a CSV with columns matching your input field names:

text,max_length,tone
"First document to summarize",100,formal
"Second document here",150,casual
"Third document",100,formal

Open Bulk Test

In the test runner, click Bulk Test or Import CSV.

Upload CSV

Select your CSV file.

Review Mapping

Confirm columns map to the correct input fields.

Run All

Click Run All Tests to execute each row.

Review Results

See success/failure status and responses for each row.

CSV Best Practices

Use quoted strings

Wrap text containing commas in double quotes: "Hello, world"

Match field names exactly

Column headers must match your input schema field names.

Start small

Test with 5-10 rows first before running hundreds.

Include edge cases

Add rows with empty optional fields, long text, special characters.

Saved Test Datasets

Save frequently-used test data for quick access:

Run a Test

Execute a test with your desired inputs.

Save Dataset

Click Save as Dataset and give it a name.

Reuse Later

Select the saved dataset from the dropdown to load those inputs.

Saved datasets are useful for:

Regression testing after prompt changes
Comparing outputs across different prompts
Onboarding team members with realistic examples

Comparing Prompts

Test the same input against multiple prompts:

Open the endpoint’s Testing tab
Select multiple prompts to compare
Enter your test input
Run the comparison
View results side-by-side

This helps you:

Compare model performance (GPT-4 vs Claude)
Evaluate prompt variations
Choose the best approach before going live

Testing with Image Inputs

When your endpoint has image input fields:

The test form shows a file upload control for each image field
Drag & drop an image or click to browse
Supported formats: JPEG, PNG, WebP
An image preview is displayed after upload
Images are automatically base64-encoded before sending

Bulk CSV testing does not support image inputs. Use single request testing for image workflows.

Viewing Image Outputs

When an endpoint generates images:

An Images tab appears in the test results panel
Generated images are displayed in a card grid
Click the Download button on any image to save it
If the model revised your prompt, the revised prompt is shown below the image

Test History

All test runs are saved in your history:

View past test inputs and outputs
Re-run previous tests
Track how responses change over time

Test history is separate from production logs. Tests don’t count against your API usage limits.

Testing Best Practices

Test Edge Cases

Empty strings, very long inputs, special characters, missing optional fields

Test Realistic Data

Use real-world examples, not just “test” and “hello world”

Test Before Promoting

Always test Draft prompts before promoting to Live

Save Good Datasets

Build a library of test cases you can reuse

Debugging Failed Tests

If your test fails or returns unexpected results:

Check Input Validation

{
  "error": "validation_error",
  "message": "Field 'text' is required"
}

Solution: Ensure all required fields are provided.

Check JSON Parsing

{
  "error": "parse_error",
  "message": "Invalid JSON in response"
}

Solution: Update your prompt to explicitly request valid JSON output.

Check Token Limits

{
  "error": "token_limit_exceeded",
  "message": "Response truncated due to max_tokens"
}

Solution: Increase max_tokens in prompt settings or ask for shorter responses.

Check Rate Limits

{
  "error": "rate_limit_exceeded",
  "message": "Too many requests"
}

Solution: Wait and retry, or reduce test frequency.

Keyboard Shortcuts

Shortcut	Action
`Ctrl/Cmd + Enter`	Run test
`Ctrl/Cmd + S`	Save test input
`Escape`	Close test runner

The Test Runner

Single Request Testing

Test Results Panel

Bulk CSV Testing

CSV Best Practices

Saved Test Datasets

Comparing Prompts

Testing with Image Inputs

Viewing Image Outputs

Test History

Testing Best Practices

Test Edge Cases

Test Realistic Data

Test Before Promoting

Save Good Datasets

Debugging Failed Tests

Check Input Validation

Check JSON Parsing

Check Token Limits

Check Rate Limits

Keyboard Shortcuts

Next Steps

Prompt Snippets

View Logs

​The Test Runner

​Single Request Testing

​Test Results Panel

​Bulk CSV Testing

​CSV Best Practices

​Saved Test Datasets

​Comparing Prompts

​Testing with Image Inputs

​Viewing Image Outputs

​Test History

​Testing Best Practices

Test Edge Cases

Test Realistic Data

Test Before Promoting

Save Good Datasets

​Debugging Failed Tests

​Check Input Validation

​Check JSON Parsing

​Check Token Limits

​Check Rate Limits

​Keyboard Shortcuts

​Next Steps

Prompt Snippets

View Logs

The Test Runner

Single Request Testing

Test Results Panel

Bulk CSV Testing

CSV Best Practices

Saved Test Datasets

Comparing Prompts

Testing with Image Inputs

Viewing Image Outputs

Test History

Testing Best Practices

Debugging Failed Tests

Check Input Validation

Check JSON Parsing

Check Token Limits

Check Rate Limits

Keyboard Shortcuts

Next Steps