I came across this use case: I want to test expensive APIs are healthy and returning valid data, think OpenAI / LLM APIs.
But each call is expensive (~cents, but it adds up if UptimeRobot is checking the /healthcheck endpoint every 5 minutes, or if someone is maliciously refreshing the page), so I added a mechanism to only test certain checks like these on a rate-limited way, currently only implementing "x times per day/hour"
Example:
# Run expensive checks a limited number of times in `allgood`
check "OpenAI is responding with a valid LLM message", run: "2 times per day" do
openai = OpenAI::Client.new
response = openai.chat(
parameters: { model: "gpt-3.5-turbo-16k",
messages: [{ role: "user", content: "Hello!"}],
}
)
make_sure response.present?
end